This is a transcript of the interview with Meng Liu and Shilaan Alzahawi from the FORRT project (Season 2, Episode 8). Listen to the podcast on Anchor. The transcript is slightly edited for readability and available under a CC0 Public Domain Dedication.
Chris Hartgerink: Welcome back to the second season of the Open Update. For Liberate Science, I'm your host Chris Hartgerink.
In this season we interviewed 10 guests over the course of nine weeks about the UNESCO Recommendation on Open Science.
A different setup in our virtual studio today, as we are interviewing multiple people at the same time, a first in the Open Update. We'll be talking today about the future of research work, especially in light of the UNESCO Recommendation on Open Science. And who better to talk to about the future of research work then the next generation of researchers.
Joining us for this episode of the Open Update are Meng Liu and Shilaan Alzahawi, all early career researchers and part of the award winning FORRT project.
Meng Liu is a PhD candidate at the faculty of education, University of Cambridge, and a methods fellow at Cambridge Digital Humanities. She focuses on the psychology of language learning, co-convenes Open Applied Linguistics and was chief editor of the graduate led Cambridge educational research e-journal, where she champions open and accessible research.
And second we're joined by Shilaan Alzahawi. She is a master student in statistics and a PhD candidate in organizational behavior at Stanford University. She's also a part of the Stanford Center for Open and Reproducible Science, where she promotes the adoption of open science practices. And for example, helps people traverse spooky things like version control.
So welcome Shilaan to the Open Update
Meng. Welcome to the Open Update.
Meng Liu: Thank you. Thank you, Chris. Thank you for having me.
Shilaan Alzahawi: Thank you so much for having me.
Chris Hartgerink: It's fantastic that you're joining here. We also had Flavio Avezedo originally planned and he couldn't make it, but we're going to have this conversation about the future of research work.
In light of the UNESCO recommendation, I want to kick us off by getting your perspective on a key question around this new policy document. You know it comes with 193 countries who say "yes, we want to support this." And throughout this series, we've been asking all kinds of people the same question, and I want to ask this question of you as well.
First Shilaan and then to Meng: So from your perspective, what is a low hanging fruit of the UNESCO Recommendation on Open Science? And what do you think is going to be a particularly difficult thing to realize?
Shilaan Alzahawi: The recommendation expresses the goal of open, scientific knowledge, which comprises access to publications, data, software code, and open educational resources
Of these I feel like open educational resources is the lowest hanging fruit.
The open educational resources movement is already well-established and has been for years. Most of us have heard about MOOCs or massive online open courses like Coursera, Edx, and Khan academy. More and more, I also see educational books like books on programming and statistics that are published in print by traditional publishers, but that come with a free online version or books that completely do without an in print version and are simply available for free online.
In terms of some of the more difficult things to realize, open access to publications has to be implemented by publishers and journals, which are often for-profit and it has to be implemented in a way that is fair and inclusive to researchers. I'm sure that many of us remember the outrage that followed when Nature revealed the terms of publishing open access in some of their journals, which includes an article processing fee for authors of over $11,000.
That's simply infeasible for the large majority of researchers and completely goes against the spirit of this recommendation on open science, which is all about accessibility and inclusivity. To me, one of the ideas behind open science is to promote inclusion and access for traditionally underrepresented groups and outrageous article processing fees just further promote inequalities in scientific public.
So we might respond by saying "let's stop submitting there and move to other journals that are more aligned with our values." But the unfortunate truth of the matter here is that so long as researchers are evaluated on traditional metrics, like impact factors, they're going to want to submit to these prestigious journals.
And that brings up what I think is the biggest challenge here, which is simply the incentives and rewards structure for individual researchers. For example, the recommendation refers to open software and code. But so long as researchers are not systematically rewarded in academia for these non-traditional outputs, then it's just unrealistic to expect uptake of these recommendations.
Software requires active and often very time-consuming maintenance. So who's going to take on this task if it's unpaid and everyone's already overburdened with research, teaching, and service committees? We need funding for the maintenance of open science practices, including those that traditionally go unrecognized by universities and promotion committees.
The recommendation talks about the economic benefits of open science practices, but it is individual researchers who have to implement these practices and who bear the costs of implementing them. So we need to be talking about the costs and benefits of open science practices for individual researchers and the potential unintended negative negative consequences of open science practices.
Like these extraordinarily high article processing charges as social scientists, we know that any intervention to assist them can have unintended consequences and we should be evaluating the impact of science.
Chris Hartgerink: That's a very fascinating perspective. I definitely haven't heard anyone in this series saying that the low hanging fruit is open educational resources.
So I would love to dig in a bit more about that, how it relates to these incentives that you say are one of the harder things to realize. So Meng I'm very interested in this twofold interview to get your perspective.
Meng Liu: First of all, I would like to thank Shilaan for a very comprehensive answer, which touches upon many points mentioned in the UNESCO recommendation. And I don't think that I need to repeat them here.
So perhaps I can touch upon a little bit of the things that are different, from what Shilaan has covered. My initial reaction to this question is that I'm not so sure if there are any sort of true low-hanging fruits when it comes to open science, simply because it is so wide ranging and all encompassing as reflected in the definition of open science in the UNESCO recommendation already.
Now, having said that, if I have to pick one low-hanging fruit, I believe it is to get more people engaged in the discussion of Open Science. While, the UNESCO recommendation is a huge milestone of progress in open science, which is a great example of open dialogue at the international level, we also need to recognize that we are still at the beginning of this journey. We're still only starting to scratch the surface when it comes to opening up research and scholarship.
The harsh reality is at the current moment there are still many people who are simply not included or engaged in this conversation and not without good reasons actually. For example, the term open science itself has traditionally been very exclusive and may discourage researchers in the humanities, for example, to engage with this construct which is why the term open research or open scholarship is often preferred as they're more inclusive.
Now, things like being mindful of the connotations of the language we use and to invite more people to be included in the open science discussion so that their voices can be heard, should be the very first steps we take. Just like the UNESCO initiated open discussions with nearly 200 member states at the international level, we should do the same at other levels as well, such as within our own research groups, our research institutions, disciplines, regions or nations.
Now again, you may have already anticipated this: I think there are many difficulties or challenges when it comes to open science, but one of the most difficult tasks I would say is equity and inclusivity in open science. And I'm not just talking about injustices, such as the digital divide. When we do open science the way we do open science now is very much focused on online platforms, open access to data and to papers, things like that. But such practices can be very exclusive and alienating for researchers who have no internet access or lack the technical skills to use digital tools.
I'm also talking about epistemic justice, that is how we can make sure that the different forms of knowledge, production and different ways of knowing are being valued equally. For example, there are many sort of guidelines or best practices being of open science, being promoted. Now when people or when we promote certain ways of knowledge production as best practices, I think we need to be mindful of the implications and consequences of such actions, to reflect on whether we are actually marginalizing certain ways of knowing and knowledge systems and unintentionally, perhaps reinforcing the inequalities and injustices already existent in the current ecosystem of science.
Chris Hartgerink: I'm intrigued now, there's so many associations I've had but this interview is about you.
How do you see the points that you've already raised in your own work, in that sense? And how does the perspective of the other influence your own thoughts right now?
Shilaan Alzahawi: I can answer two questions at the same time.
So first of all, you seem surprised that I brought up open educational resources as a low-hanging fruit. And then Meng also mentioned that she thinks that there maybe are no such thing as low-hanging fruits in open science, that there's a big burden on researchers. It's a lot to ask and it's a very diverse multicomponent process. And I think it's very fair to get pushback on this specific point. And I also want to respond to what Meng said about diversity.
So first let me mention open educational resources. One reason why I think this may be a bit more of a low-hanging fruit and why it has already seen quite some implementation, is that teaching is more traditionally aligned with what promotion and tenure committees take into account then some of these more novel proposals, like open source software and open code and data. There's already a strong existing infrastructure for open educational resources and we all do it. We all teach. I think it might be easier to just take what we already do and start sharing it openly so that you can now put it on your CV, that your online course, or your book has had this many clicks and this many likes and this many downloads.
But again, I think it's very fair to push back on my point and to ask "might it be naive to see this as a low hanging fruit?" This brings me to another example, which is open data. In the recommendation, the reuse of data seems to be described as a universally good thing. If you'd asked me about open data a week ago, I would have said "sure, that's a low-hanging fruit, everyone benefits, and everyone is happy."
But thanks to academic Twitter, I realized that this perspective was perhaps a bit naive and that there actually are a number of researchers who worry about early career researchers, spending years collecting data and accessing hard to reach populations. All to have their open data be reused without citation or attribution.
I think that's something we have to be mindful of. This goes back to Meng's point about diversity and inclusion. We have to think about the diversity of disciplines and how it's impossible to implement a one size fits all approach to open data and to open science more generally.
In some cases, data collection is costly or sharing data is difficult due to ethical issues intellectual property concerns, or a lack of resources. In my field specifically data collection can be quite cheap and easy if you're willing to rely on online participant pools. But of course there are other fields in which it is much more time consuming and costly to collect data. And it's impossible for me to say how I would feel in that situation about my data being freely used by others without proper citation or attribution.
Chris Hartgerink: Could you share a bit more about what happened between last week and this week? I hear you say Twitter.
Shilaan Alzahawi: I'm not sure if I actually changed my own opinion on this, but at the very least I was confronted with the reality that there's no consensus about it.
I think that if we would have had this interview a week ago and you would have asked me how people feel about open data, I think I would have said something about there being a consensus.
The truth of the matter is that I'm impacted by the social environment that I'm in. And I do research in social psychology and organizational behavior, and most people around me collect their data from MTurk, from Prolific and from other online participant pools.
I'm simply not confronted with the reality for some researchers where it's really hard for them to collect data. And that's what this Twitter discussion made me realize. I believe that there was a woman who just openly asked someone's perspective or academia's perspective on the fact that someone had used their data without reaching out.
And of course there were a bunch of quote tweets and responses where you could really see a diversity in the replies in the responses. And that was a wake up moment for me that "no, not everyone agrees on this. Very understandably for me, it was very insightful to realize that we don't all come from the same place.
We don't all have the same resources. We don't all take the same amount of time to collect data. And so for some people it's simply is a bigger ask than for others. And that is something we have to take into account.
Chris Hartgerink: It's very interesting. Meng, you just said that the hard part is having those discussions and now I hear Shilaan already saying these discussions are being had.
Meng Liu: I'm not saying there are no discussions happening. I'm saying not enough people are getting involved in this kind of discussion.
I have to say, unfortunately, that the current discourse on open science is very much quantitatively biased is just humanities researchers or, for me like a social scientist we see the term open science and we don't immediately feel that's relatable. And especially for the humanities. I think we really need to think about ways to improve this.
For example, changing the terms and trying to initiate a reconceptualization of how we define open science with regards to different disciplines, to different epistemologies. So that they can feel like, okay, they're welcomed, that they're valued and they should be part of this discussion. So I think that's what I'm trying to push for.
I'm also glad that Shilaan mentioned open data, because I think this is a super fascinating topic. This is one of the topics that, that the sort of the disciplinary differences are most salient. You really see how people, subscribing to different research paradigms, wield this open data issue differently. For example, for certain sort of qualitative research paradigms, it is not just about the practicality of making data open, but whether like why should we do this in the first place? And what are the consequences and implications of making the data open?
It could very well, for example, change the dynamic between the researcher and the participant where the participants may no longer feel like this is a safe space for me to, fully express my views my opinions and that can change the kind of knowledge we will be able to produce from that type of research.
So in that sense, this is even in some cases, antithetical to our original goal: we wanted to increase the research quality. We wanted to make sure that we are generating valuable knowledge. So in those cases, the issue of open data is really more profound or touches upon some of the deeper issues than just the practicality or the technical ability, for example, to anonymize data.
So it goes much deeper than that. And I think we should welcome and invite people from those research backgrounds to contribute their expertise to share with us, how they feel about these kinds of issues. And can we reach a consensus on what are the common values and what are, what might be some of the feasible ways that we can get there?
Chris Hartgerink: So do you feel like for your careers, that the work that you're doing, and of course this is a feeling less a knowing because you are in early stages of your careers, do you feel like this is going to pay off?
Meng Liu: I wish I could say the incentive system is changing fast enough for me to enjoy the benefit. But unfortunately I have this feeling like, this type of change does take time and especially for people doing this.
I think we're already in a slightly better state than people in early years. For example, I can't imagine the kind of pressure they were under or the struggle they had to face, between their personal career development and the sort of the values they hold dear. So the misalignment between, what is good for the scientist and what is good for science was even more severe back in the days. We're already in a slightly better position nowadays but I would say, some people are more lucky than others. They are in a field that's more open to, to open science, that they're changing things faster. And in other fields, this type of changes are still in very early stage. And it may take many years before, everybody could reap the benefit of that. This is what I believe in this is. Aligning with my personal values.
And I see that as a way for me to keep my faith in this whole endeavor. So of course I need to face the realistic demands or expectations on me to publish in certain journals, or publish a certain number of papers each year or something like that. But in the meantime, I also think it is important to do what you believe in.
Chris Hartgerink: Shilaan, do you feel like it's like it's paying off for your career?
Shilaan Alzahawi: To be very frank, I'm not sure my open science activities pay off directly. I don't think I will be more likely to get a job or be well evaluated because I take time out of my day to diffuse open science practices. I think some people around me wonder why I'm not spending that time doing research.
But it does pay off for me indirectly, in my opinion one in the sense that I think and hope I'm becoming a stronger researcher through open science practices. So what I learned through open science influenced my experimental research. And second, I completely agree with everything that Meng just said about aligning with your personal values.
I like my work much more this way, to be honest. I started my PhD and experiments used to give me a massive amount of existential dread. I found them scary because I felt like there were an infinite amount of moderators and explanations for what I was doing. I felt icky about the degrees of freedom I had.
There were just many things that kept me from enjoying my work and it wasn't until I started implementing open science practices into my own work that I noticed, okay, this feels this feels a bit more true to how I want to do research. And it has really made me much more motivated and productive as a result.
So at the very least, I think that open science may have indirectly benefited me.
Chris Hartgerink: You especially, you've been working in big team science projects, and it seems like both of you are very excited about it. And I wonder whether some of that excitement might rub off on our listeners, because I know that it's still a rare thing for people to be a part of a big team science project, whether it's because it's too much work to apply for funding with multiple institutions or for other reasons.
Shilaan Alzahawi: I think I've gotten two things primarily from doing big team science projects.
One is the community, to be honest. It's amazing to be connected to hundreds of researchers at different institutions in different continents. The second thing I think, more of a benefit that I've perceived from doing big team science is I feel like when you're doing experimental science, you often get pushback when you're presenting things or when you're publishing papers on, the boundary conditions and to what extent it generalizes, across different cultures and different countries and different languages.
To me that's been truly amazing, the fact that we can finally start answering some of those questions. Those are really big questions, whether the fundamental attribution error works the same way for every single individual on earth, which is what we used to assume, even though we used to only run those studies on north American college students between 19 and 23 years old, who are highly educated, non-religious et cetera, et cetera. Now we can start taking a look at moderators and demographics and how they impact our research findings.
For me, team science has really been a way to rebuild the foundations of the field that I'm in and to find out to what extent those foundations actually hold, which has been fascinating to me.
Meng Liu: First of all, I have to say that it really resonates with me a lot, what Shilaan just mentioned, especially the, the sense of community. I first got involved with the FORRT project near the beginning of the pandemic. That's when I started, using Twitter more and stuff like that.
As you can see, this kind of involvement actually not only deepens my knowledge about the research process and what science means, et cetera, but also, helps me to gain a sense of connection with the world that's out there. That really helped me with my mental health as well during this very challenging, past three years or so.
So that's definitely a big plus for a big team or for a community, especially. The early-career communities such as FORRT actually brings me to another point. I feel like I'm personally interested in research methods and I think mass collaboration or crowdsourcing can be a really valuable research method that can help us advance knowledge in ways that are different from, how it was traditionally done.
At least in my field. So I think that's why I've been involved in quite a number of projects in FORRT because I wanted to learn w what are the sort of benefits, what are some of the pitfalls or challenges in this kind of model and how can we optimize this model and how can we apply it to our own research to our own research field.
Chris Hartgerink: The podcast ends, and then the day continues. What will be one practical thing you would invite people to do? So that by the time they've heard from both of you, that they'll be like, "ah, tomorrow I'll try this."
Meng Liu:
Shilaan Alzahawi: So if we go back to the community aspect of this, I think Meng and I will agree on this.
One thing is to join the FORRT Slack. That is the one way I think I've become know part of so many big team science projects, and this is just the resources out there, the support, the community. It's wonderful. So it's it's one very easy step that has, I think, consequences for a long time.
Chris Hartgerink: Okay, so join FORRT; Meng, what do you add to that? Another join FORRT – so just do it twice?
Meng Liu: First of all, another join FORRT!
The other thing, I think there's a lot of discussion happening on Twitter. I think Twitter is the place like where many of the, sort of the discussion on open science is, compared with other sort of social media platforms.
I also think when it comes to research, for example, you don't need to think "okay, oh my God, I need to do a replication study" or, "oh, I need to, do this register report. That's gonna, take rounds of review or front loads, everything. And it's going to be drastically different for, from how I learned how to do things."
You can just start by sharing preprints or postprints. That can be a really small step, but can ease your way into this whole world of open science. Don't try to do everything in one go and start really small.
You just listened to an original interview here on the second season of the Open Update with Meng Liu and Shilaan Alzahawi.
What did you think? What insights came to mind and what resonated, but also what did you disagree with? I would love to hear from you our listeners of the show, because this isn't just a conversation on a podcast. It's a conversation for us all.
So leave us a voice message. It doesn't need to be a perfect voice message. By sending us a voice message, you'll also take part in our lottery to win a copy of the book Ways of Being by our previous guest, James Bridle. Be sure to include a way for us to get in touch with you so that if you win, we can actually contact you.
For now, have a good rest of your day. Take it easy. Next week, we'll be back with our final interview of the second season with Kaitlin Thaney, where we'll talk about how we can think do and build open into our every day to improve the future.
