🔈This is the transcript of the Open Update. Find the original audio on Anchor.fm.
[00:00:00] Chris Hartgerink: Hi, and welcome to the Open Update. We're here for another episode. For Liberate Science, I'm Chris Hartgerink, joined by my lovely co-host, Sarahanne Field.
[00:00:10] Chris Hartgerink: In the Open Update, we talk about power imbalances in research and the societies we live in to also start better understanding what we can imagine as a realistic alternative.
[00:00:22] Chris Hartgerink: Just like in Star Wars, we wanna bring balance and the first step is to make ourselves aware of these power imbalances. So thanks for joining us on this journey.
[00:00:32] Chris Hartgerink: Today we'll talk about gamification of researchers, what it does to the power balances, and also what the effects are more broadly and maybe some of the history within open science and open research, some events that relate to the gamification and discussion around it.
So Sarahanne, you said this was on your mind lately and you'd love to discuss this. For our listeners, what is gamification and what does it have to do with power imbalances to begin with?
[00:01:03] Sarahanne Field: The basic idea is, is to motivate people to engage, by using game elements, competitiveness, aspects of fun, talking about rules and particular ways of playing. As it relates to open research, it's basically the idea that you are using fun elements of play and of games to motivate people to engage with open research.
[00:01:25] Sarahanne Field: Responses to gamification in open research in particular are mixed. Some people think it's really fun and think it's good, especially for getting earlier career researchers to get involved, but some people find it infantilizing or patronizing.
[00:01:40] Sarahanne Field: So, for example, a lot of people are aware of the badges that some journals use for rewarding things like open data, open materials, pre-registration, so you can get badges, which are sort of a kinda a fun element of engagement.
[00:01:55] Sarahanne Field: And again, that sort of is an attempt to motivate people to try and these badges, another more sort of controversial example of gamification is the curate science leaderboard.
[00:02:08] Sarahanne Field: So leaderboards play a key role in gamification, because they try and get people to compete against one another. But it's, it's also been a little bit controversial because people feel, that it's, a bullying, aspect to it or that we shouldn't be inducing competition in research because we already have that problem of it being too competitive a culture, and it had been toxic as a result of it.
[00:02:34] Chris Hartgerink: I like that you mentioned different examples because talking about gamification as a whole, it might result in discussions that might not appreciate the variations. This idea of inducing competition with the leader board can be very detrimental. Whenever I see a leaderboard, I, one always feel incompetent. Like if you play a game on your phone or something and then you get a really good score, you think, and then somebody else has a score that's like a hundred thousand times higher and it always makes me feel very incompetent in it.
[00:03:05] Chris Hartgerink: But then this idea with, with the badges is, "oh, I'm trying to collect them all", "what do I have to do", and "how can I unlock certain things."
[00:03:15] Chris Hartgerink: It's fairly interesting to maybe chat a bit about this history because, especially in terms of, you know, gamification in open science. You mentioned badges and leaderboards, which have two very specific histories. I think with the badges it's been. This idea of, you know, does it really work or not to incentivize people to share?
[00:03:36] Chris Hartgerink: The evidence has been disputed, in that sense.
[00:03:38] Chris Hartgerink: And then with the leaderboards, there's also this idea of, who chooses to participate in them. And especially the example of, curate science, which went overboard in that sense.
[00:03:53] Sarahanne Field: There's so much to sort of unpack, right? I mean, one thing that I think about when I think about gamification in general and also in particular leaderboards, is whether the motivation is intrinsic or not. so the literature tends to sort of point towards leaderboards in gamification as sort of not really increasing intrinsic motivation.
[00:04:13] Sarahanne Field: And I think that's a problem for something like, Increasing transparency, increasing those kinds of sort of better practices. It's important to emphasize elements of gamification that do increase motivation, but also increase intrinsic motivation.
[00:04:30] Sarahanne Field: You wanna increase engagement , but it's important to increase that intrinsic motivation. You're playing the game to really engage in the practices rather than playing the game just to win. I think it's important to balance these two when it comes to changing practices and changing culture. I sort of think about this, you know, how, what role does it play in changing culturally, just becoming more competitive?
[00:04:53] Sarahanne Field: Are we just looking at these motivations that sort of tie into our image and our profile? Or is it really changing practices and, and really shifting culture? You know, and that's what, that's what concerns me. So I guess that's, this is sort of a more negative thing. Um, but, but that, I think there are plenty of positives, you know, people who are entered the, the gamification.
[00:05:14] Sarahanne Field: Stuff. You know, I think it can be really good. They can get a, you know, a scientific record for themselves that in which they get, they're aiming for getting all the badges, and I think that ultimately has a really good impact on science because if they're opening up their research more as a result of it, if they're preregistering or, or doing a registered report more as a result of it, then I think that's ultimately for science, for research.
[00:05:40] Sarahanne Field: I think that's a really good thing. So I think as long as we are balancing, you know, these different elements and keeping, keeping thoughtful and reflective about these things, rather than just doing it for, for winning, I, you know, I think it can be a good thing ultimately and net and net positive I would say.
[00:05:57] Chris Hartgerink: In that sense, science is already very gamified. With the journal rankings, that's sort of a leaderboard aspect, right? How many publications do you have? Where have you published? It makes me think of what is the purpose of a specific instance of gamification.
[00:06:13] Chris Hartgerink: So with journal rankings, I think that doesn't work because the purpose is there is no real purpose except for being in competition. There could be another form of gamification, which has the purpose of, for example, connecting researchers more, participating in communities, if there's certain, you know, if we check in once a week in a certain email list, and that gives you certain quote unquote rewards, then that's a form of gamification, which has the purpose of connecting people and sharing information, which I think is a completely different aspect. So I guess coming back to this question of what is the purpose of a specific instance of gamification is incredibly important because otherwise it ends up being self-defining, like for it for its own sake.
[00:07:03] Chris Hartgerink: With badges for example, the purpose is to increase the behavior of sharing data, for example. And I think that's in essence good but then, and indeed, as you say, how can we be reflective of this? Because if we're promoting that behavior of, for example, sharing data, are we then people who cannot share data, are we making them quote unquote lesser than. So this idea of thinking about what are the people who cannot get rewarded? Is it something really that people can participate in if they want to, or is it more a circumstance? In that sense, it makes me think of if it ends up in the right behaviors, then it's warranted. Makes me also think of, you know, the ends do not justify the means, and what is the process in gamification and is it really something we want to encourage.
[00:08:09] Chris Hartgerink: There this idea of that it's very patronizing or paternalistic because the person designing the game is saying, this is the outcome we want and we're gonna make sure you go towards that or the probability that you go towards that is higher.
[00:08:25] Chris Hartgerink: So in that sense, I guess also this question of who, who's designing the games and do we have insight into These as well, because, everything in life is a game. Especially over the past 15 years, I've definitely noticed that my attention is being gamed in many ways that I really actively have to avoid Instagram reels or TikTok or YouTube shorts because I end up spending way more time than I want to. That's also a form of gamification.
[00:09:00] Sarahanne Field: One thing that sort of was a concern about the curate science leader board, and this comes back to the idea of power imbalances and something you just touched on before.
[00:09:10] Sarahanne Field: You know, who, who's able to participate and whatnot. There are two things I wanted to mention. First thing is I think that the gamification is less harmful when people are able to engage at will. So the Curate Science Leader Board, which, which some people call the Transparency audit, which is a very different framing, that gives it a very different twist.
[00:09:33] Sarahanne Field: People had to ask to, to be taken off it. So people were being put on this leaderboard without their knowledge. And so some people were asking to, to be taken off it cause they didn't agree with it. There's a problem with putting people up on a board without their consent. For something like this, I think being able to opt in is important there because you are then agreeing to be part of that game, right?
[00:09:57] Sarahanne Field: People should have the choice that the right to be able to sort of opt in for that rather than having to opt out.
[00:10:04] Sarahanne Field: Coming back to what you said before, you know when people can't, for example, share data. If you are, for example, like myself, I do mixed methods research and depending on what I'm doing on a given day, some of that is just not, not shareable. I, I cannot share people's sensitive personal data. That's, that's a legal thing as well as a moral thing, and. So, you know, by default some of those aspects I'm just not gonna be competitive in.
[00:10:28] Sarahanne Field: I would definitely like to have a say in, you know, whether or not I will be on that board because I don't wanna be compared to people who are doing, for example, only quantitative research that can be completely anonymized and who are naturally by default, just by their methodology, are gonna be able to be more competitive.
[00:10:45] Sarahanne Field: That's a bit of a concern of mine and it comes back to sort of what the dominant traditions in research are right that the quantitative science tends to be dominant. That, you know, white perspectives tend to be dominant. I think we have an issue there too.
[00:11:01] Chris Hartgerink: You inspire a bunch of thoughts and I think one that I want to pick out there is that, what you say, there's this first order. The very first level of, you know, what is the game you're playing and are those rules even transparent to you?
[00:11:18] Chris Hartgerink: But then there's also this second order reasoning there, where is it even clear whether you're playing the game that you want to in that sense. So the idea of, you know, It depends very much on the trajectory that you want to be in, whether you want to play game A or game B or none at all. And in that sense, this idea of there's a universal game all researchers need to play that's gonna be incredibly problematic to begin with.
[00:11:50] Chris Hartgerink: And maybe. Maybe we need more games, so more gamification so people can actually make decisions as to I'd rather participate in this and not in that one. So really more alternatives to journal rankings. More alternatives to publications. I did not expect this to go into assess systems, but in essence, an assessment system is a form of gamification.
[00:12:15] Chris Hartgerink: At the same time, I love playing video games and it can be very encouraging as well. So I would, I would even encourage more thoughtfully designed games or gamification that can actually move the needle in the right direction. But maybe that isn't a good idea anyway because, What is the right direction?
[00:12:40] Chris Hartgerink: It's not just the effects of the gamification, but it's this question of do we know to what purpose something needs to be gamified. Research equals specifically, we also think about gamification of, how do we get people to just enjoy what they're doing, not even be in competition, but to simply have this be, process that's enjoyable.
[00:13:03] Chris Hartgerink: And where they can also, you know, if they put time in that they can get something out. And I think with education very often, it's gamified in a way of you receive a certificate. And you can showcase that somewhere else, and it gives you access to quote unquote, the next level in your career. I think a master's degree or a PhD is sort of, you know, you're, you're leveling up.
[00:13:25] Sarahanne Field: One thing I'd like to just touch on briefly that you mentioned earlier is that who is creating these games?
[00:13:32] Sarahanne Field: So often we have people who are already prominent in the open research community, for example, who are the ones who are heading up the ideas for these games, who are producing the rules, who are, who are basically saying, you know, what the game is about and what it's for.
[00:13:47] Sarahanne Field: The more voices that we get involved with gamification, the more priorities and needs are going to be reflected in these games. And I think that's important. It comes back to sort of whose voices are we hearing in the open research community and whose perspectives are sort of represented. The more diversity we see in open research, I hope that the more diversity we're gonna see in the gamification of these different practices, I think that's, you know, something that's traveling in the right direction and that we need to be critical about when we engage with a game, Being critical about whose voices and whose needs and priorities are being, are being reflected in that game and taking that into account and reflecting on that. I think that's important.
[00:14:30] Chris Hartgerink: I keep flip flopping during this conversation cuz I think gamification in a way is also a very production driven, like optimizing output driven mechanism. Because it makes people expend more energy and it reminds me of somebody sharing in the, in the signal group as well. This whole like toot on how we're not discussing enough how improving research relates to anti-capitalist behavior as well.
[00:14:57] Chris Hartgerink: Where it's not just about increasing efficiency and reducing research waste, which is within the improving research communities, substream, which I agree with the premise, but it also reinforces this idea of we be need to be optimal research producers. And knowledge isn't always, or understanding isn't always produced in the the most optimal way.
[00:15:25] Chris Hartgerink: So in that sense, you know, if gamification serves that purpose, I think that's an angle where I would be hesitant.
[00:15:36] Sarahanne Field: Really thinking about what the purpose currently is and maybe what it should be and what the difference is there. So I think, gamification is traditionally used for motivation, right? It's traditionally used to, to increase production, to increase productivity, to do things better, to optimize, like you said. But indeed, I think one, you know, purpose that we can maybe try and engender is, is the purpose of having fun and enjoying research and learning, connecting with each other, as you've already mentioned.
[00:16:07] Sarahanne Field: Those purposes to me are, not yet evident in, in what I see in gamification and open research. That's something we can really improve on
[00:16:16] Chris Hartgerink: Well, I think it's time for us to move on to final remarks.
[00:16:20] Chris Hartgerink: For me, a final remark that came to mind is also that gamification can be used for equity if it's done right. I always love this in games where you get this, this in-game currency, which you can exchange for certain perks or benefits. And I feel like if we would set up the purpose of gamification is to indeed progress in the direction that you want to in your career.
[00:16:46] Chris Hartgerink: That we can reward certain behaviors. So we could say, teach a course that's. I don't know, 200 points. And if you complete a study that's, for example, also 200 points, literally saying, Teaching and doing a study. So teaching and research are worth just as much. And then if you get like, I don't know, 3000 points, that's when you unlock your tenure.
[00:17:11] Chris Hartgerink: And that way to also very clearly say, it doesn't matter whether it's a, a man or a woman or a black person or a white person who unlocks these points. Once you have them, no matter how you got them, you can unlock that next step. And I think that, so there's also a potential there, but very much depends on the design and who makes those decisions.
[00:17:36] Sarahanne Field: I would just say that if you're, if you're going to be engaging in one of these, you know, examples of gamification, I think just be critical. Reflect on who's been making up the rules, who it might include and who it might exclude, and just be thoughtful about how you wanna involve your own self with those, with those kinds of games.
[00:17:56] Chris Hartgerink: That's us for this week's episode of the Open Update. Maybe you know, of games that you're a part of in your environment or rules that have become clear as you've gone throughout your career. So join us in our signal group. Let us know what those might be, so we can share them out with the rest of the listeners because a lot of gamification ends up being implied and becoming aware of this is also incredibly helpful to be able to address them and start imagining those alternatives, which is exactly what we try to do here at the open update.
[00:18:30] Chris Hartgerink: We were very happy to see that there was quite some response to this idea of alternative forms of peer feedback.
[00:18:36] Chris Hartgerink: Who knows maybe that could become more of a series where we can start thinking about, well, what are alternative forms for x. And so if you have ideas about spaces where you like to see alternatives, whether that's alternatives to funding, alternatives to study, design or whatever.
[00:18:54] Chris Hartgerink: Let us know. Join our Signal group, leave a voice message. And who knows, maybe we'll cover it . With that said, thank you for listening to this episode. If you would be so kind, please do leave a review on Apple Podcast if that's your thing. Don't forget to share the podcast with some of your colleagues or friends or your grandparents, who might not really want to listen to this.
