Onboarding with confidence in open research (s03e16)

🔈This is a transcript of the Open Update podcast. Listen to the original audio on Anchor.fm.

[00:00:00] Chris Hartgerink: Welcome back to the Open Update.

We're back in the series talking about the decade of open science in the last 10 years, and then also this week we're gonna be talking a bit about, what do we foresee for the next 10 years?

In the last episode we talked about is open science stagnating, how do we look at the developments over the past 10 years in various areas? Of course, open access, we have very clear numbers that it's gotten much, much better. But open science is much more than just that. So there's also policy that's in place now that is promising for the next 10 years, but we want to take time today to also talk about what is it that we receive for the next 10 years? Where are some opportunities, some challenges, and what are some of the risks in terms of how we've experienced the open research movement in the past 10 years? What are some things to look out for?

[00:00:54] Sarahanne Field: Something that I'm excited about seeing the development of in the coming years is, is the critical branch of meta science where we're not just accepting that these forms of openness are beneficial to everyone in every context.

But considering some of the nuanced aspects of that when pre-registration is not valuable or beneficial, when registered reports are problematic as a format, when openness doesn't always work for the benefit of everyone. So I'm, I'm really interested in where those discussions are going to go. That's a branch of meta science that I'm starting to see taking shape.

It's one thing to, to criticize other people's research as we have been doing for the last decade with replications, with calling people out with retractions. But now we're moving on to, "okay, so how does the tenor of that discussion go?" How does that tone, how should that tone look? So that's something that I'm excited to see sort of opening up in the future.

[00:01:54] Chris Hartgerink:  For me personally, one of the things that I did not think would happen, at some point you just notice it's the same old discussions all over again. I'm still hearing these conversations about the impact factors, even though we knew about this eight years ago, just as well. Well, it's maybe a few different people having this conversation, so I understand it's also a bit of onboarding probably for people entering this space.

But why is this even still important? We're all susceptible to that feeling of we're having conversations all over again. That is something I definitely wanna look out for, is to really push that conversation forward about what does it mean to improve open research and actually making these things happen instead of just talking about them.

[00:02:38] Sarahanne Field: I'm really interested in these unintended consequences of the science reform movement.

That's another sort of thing that I'm really getting excited about, because often we are really uprooting parts of a larger, very complicated moving system, and so understanding, you know, what some of the challenges and the, the consequences of that are, that's a sort of a branch of, of critical meta science that I think is really great.

[00:03:02] Chris Hartgerink: It's very interesting to see that change happening because we talked about solutionism and trying to fix problems, and I think from that framework, unintended consequences are unforeseen very often, and of course unintended consequences are very often things we didn't think about, but it's exactly that point where if we stop thinking about fixing one problem and then, you know, clearing ourselves and saying, okay, we're done here, then those unintended consequences become part of the process.

It's the, "okay, we address one thing and something else pops up." It's like a continuous whack-a-mole of just trying to move forward. So in that sense, I think those two things are very tied to each other 'cause only if we really have a way to have those conversations in a productive way, to express criticism, to express concerns, to express just thoughts around specific ideas, only then can we identify where that next step might be taken.

In those next 10 years, if we were to do an episode like this or a series of like this, again in 10 years, I would be very, very glad if we can look back and identify, okay, we've actually made much more decisions, much more statements, much more, joint efforts in terms of how resources get spent, how power gets used, how we organize things. Because right now we're very often making those decisions individually, or we have open letters where we sign on to a statement, which is also a form of organizing.

But on the scale of organizing, I think that's an entry point. It's a relatively low hanging fruit instead of, you know, completely the opposite. How do we actually organize to complete a project? 'cause imagine if we were able to do a large scale project with a hundred people, you know, replication projects, they've done this. How would we do this on a continuous scale to really, you know, mobilize beyond. Just a research project. I think that's something that I would be very keen on being able to look back on in 10 years.

[00:05:30] Sarahanne Field:  Just wanted to touch on something you mentioned before about how we can't always anticipate the next step because often these consequences are unintended. This comes back to sort of something I've been thinking more and more about lately is that open is a such an enormous toolbox of practices. When I'm thinking about anticipating the, the possible chains of effect of certain actions, what comes along with that is also accountability.

For me, looking forward, you know, it, I would like to see people engaging with the challenges of open research with this kind of multifaceted approach. Having this anticipation, having this accountability alongside the actions that they are taking. I think that that rounded approach is, is gonna be beneficial and that's gonna really be valuable. That's, that's my perspective.

You mentioned that part of what causes a stagnation, that we're seeing is partly onboarding. So by that I'm assuming you mean when new researchers, for example, are, are coming on board with, with open research, they're learning the discourse, they're, they're learning to contribute their own ideas.

'cause this is gonna be constantly happening. I know people literally every day, every minute are coming to open research. How do we allow that onboarding and encourage that onboarding? 'cause of course we want this without getting redundancy or getting stagnation. Do you have any thoughts on that?

[00:07:07] Chris Hartgerink: I always like when people say, let's create onboarding ramps, because I think that's a very natural way of thinking about it. You can't just immediately hit the highway without an onboarding ramp. Right. I didn't necessarily want to criticize that these conversations are being had. 'cause I think it's more the, that there's still a need to have these conversations.

So in terms of onboarding, there's "one" too many places where there's literally zero onboarding. Here's your key to the office, here's your laptop or your computer, your logging info, and you know, figure it out on your own. Especially with early career researchers who are entering a PhD.

You might have situations where it's really nice and there's actually a group who cares about that, and they also go beyond the onboarding they might have, for example, you know, building trust within a team, which is incredibly important.

And then also offboarding to understand where our improvement factors for that specific group. Having both the conversations at the very beginning, at the very end, but also intermittently in between to understand what's happening. And I think there's just too many places within research where very normal management practices aren't implemented at all.

So for example, if I were to enter a research group today, I would very much like to know, what is the culture? What is the expectation with respect to how do we communicate? What is the expectation with respect to how do we work within a team?

You know, when something gets submitted, is it enough to have people's implicit approval or explicit approval? And just setting up that cultural norm. Then also vice versa. Literally saying, okay, this is what we don't do. So if there is a group where you come into and you know, journal impact factors are not important at all, but everybody else at the department talks about them, then you're gonna be the odd one out and it's gonna influence how you see it.

So I think just in that sense, onboarding is incredibly important to not necessarily preempt these conversations, but to create clarity. Because I think I've mentioned this in another episode like way back, but in ambiguity there resides a lot of power. And I think with a lot of research groups, there's just incredible amounts of ambiguity.

There's a meeting and then there's a decision, and I feel like that also creates inconsistencies between person 1, 2, 3. In the same research group, and if there's one thing about good management is that you apply it the same way to everybody who is affected by that, regardless of whether they're an, an incredible say, PhD candidate or an average one, or, underperforming one.

I think in that sense, there's just so much we can do to make sure that management. Creates good conditions. The compost for these practices to flourish.

[00:10:16] Sarahanne Field: That's what I really like about what FORTT are doing. They are basically a landing page for people to go as sort of a first step in their open research journey. I love that because it does help people, in the sense that it provides a uniform basis for everyone to sort of start on. It provides information, templates, that kind of thing. And it really does help with onboarding so that people aren't sort of reinventing the wheel for themselves. That really helps with, especially with, with ECRs who are still finding their footing in research in general.

Learning, you know, to conduct research. And then on top of that, this extra level of methodology and, and philosophy to sort of also learn that alongside.

[00:11:00] Chris Hartgerink: FORTT is doing tremendous work. Just the size of that community and how fast they've grown in a decentralized system. Flip side of FORTT, but also a lot of other onboarding or in general academic approaches to communicating is, it tends to be incredibly precise. And incredibly lengthy and incredibly difficult to understand.

It's not an onboarding language very often, so I think that there's still room for improvement in terms of how do we write, with what clarity do we enter that situation and with the purpose to communicate something effectively in different forms, but also in a succinct way.

I often stumble upon resources that are pages and pages and pages long, which creates a lot of attrition, which creates a lot of, lack of retaining, even though maybe if you can summarize it in, say, a few paragraphs that are very well copywritten, that aren't perfect. If retaining the information is higher, I think that trade off is worth it.

So this aspect of getting out of our academic mindset sometimes is very difficult. But it's important because I see that there we could really, really gain a lot of ground. But that is my personal opinion on that front because I only started noticing how much that influenced how I personally wrote after I stopped needing to do it. Because clear communication really removes the need for so much extra work and clear communication is incredibly difficult. Let's be, let's be honest.

[00:12:49] Sarahanne Field: I mean, when you come into the open space, there is so much to learn.

So say you wanna conduct a replication, you know, there are lots of different templates for how to conduct that. There are a lot of different ideas on how you might wanna go about choosing a study for replication. There are different ways to pre-register your plans for a, a replication, for example.

Having piles and piles of resources to go through is a huge amount of labor. But I think it's, it's difficult because there is so much information to be consumed, but that's why I really like Christina Bergman's buffet approach to open research. For the listeners who aren't familiar with that, it's this idea that when you approach open research, you pick and choose the approaches or the resources or the tools you use, which fit best into your way of doing research. Into your particular research paradigm, into your particular epistemology.

And I think that's really important to help educate people on that way of thinking about open research. It's not about you have to adopt all of the things, do all the stuff or you're not an open researcher. It applies to you in whatever phase you're at, in whatever way you do research. It's a matter of helping, you know, guide people to what's most useful for them and how they can contribute to the open movement, if you can call it.

[00:14:12] Chris Hartgerink: Yeah. And in that sense, also, what, what helps reduce the complexity is clear language, of course. But there's also a, you know, recognizing indeed that you don't have to eat everything from the buffet. You don't need to finish all the plates. Um, and there's something for everyone. At the same time also reducing all of this complexity to fundamental principles.

I think we too often get sucked into talking about reproducibility, replicability, open access, APCs, all of these specific things, when in essence all of it can be really drilled down to working openly. And that doesn't mean everything needs to be open, but this idea of can others access your results or whatever it is that you're doing, and understanding where in that process it might not be possible.

And just reflecting on that and saying, okay, what could I do to improve that bit? For me, that that has been really the, the red line throughout my own work. It's the. I'm reviewing something. Can I understand what they, what they did in the paper, where those results came from? Oh, I'm submitting something. Oh, you know, can people actually access the data? I might have a link in there, but let's double check. And so all of these things about working openly, it's also this question of just simply what is the, what is the narrative around your own work, and is that publicly available?

Sometimes you can actually decide. I don't want this to be available, but to me that has really been the core of my own philosophy. And then it's no longer about applying that in research, but it, it could be applied anywhere. Whether, whether I, you know, if I go to software engineering, of course we have open source, but that same principle applies if we go to finances. You could ask the same question. If you go to legal scholarship, you could ask that question as well. Of course, it gets different answers. But in that sense, reducing that complexity is, I hope that that happens in the next 10 years and that, uh, that that helps make it easier to navigate and that people also get the encouragement to discover what that means for them, because in that sense, Anybody's working process, regardless of whether it's open or closed or anything in between is very personal and changes all the time.

[00:16:49] Sarahanne Field: What I like about some of the maturity of, of the movement that we're we're coming into, is that we are interrogating that assumption of open always being best. Where some of us at least are getting the confidence to sort of decipher ourselves rather than just drinking the open Kool-Aid. We're starting to think about this more actively engaging with that question of "when is open most beneficial and is, is it always beneficial and for every case?"

That's something that I try and instill in, in people who come to me for, for advice, students I have, for example. I try and just instill a sense of confidence in them and give them the idea that they should be conducting research in the spirit of openness.

So they may not have all the skills and the knowledge to engage in all the practices they want to. But I think what has to come first is acting in that spirit of honesty and transparency and integrity. That's a really important place to start and have confidence in acting in that way, and in, in my opinion, so much falls into place after that.

Then you can learn, for example, as just an undergraduate student to write your research reports with transparency, be aware of things like supplemental material and that they can be published alongside the paper. Encouraging detail when it comes to methodological reporting, for example, and giving them confidence that they can just start at that level.

So I think that's something that I really like, is that we're just starting to sort of see how we can act in the spirit of openness and not always have to be doing all the things. That's sort of a natural progression from Bergman's idea of, of the open buffet is just starting off, you know, as you're onboarding in general or as an ECR coming into open research, that you begin with acting in the spirit of that.

It takes time, right? To build the skill to perform these practices properly. It's difficult to get all of the steps right. And there is a fear in a lot of, especially ECRs, that if they do it wrong, the open science bullies are gonna come for them. People have said this to me that they're afraid of that. So I think instilling that sense of confidence and, and getting them to, to learn to think in the spirit of openness is a great place to start.

So, confidence building. Capacity building. Giving them the, the self-efficacy and the confidence to, to think I can build these skills up. I have the space and the time to learn these things, rather than saying, yeah, all the practices have to happen like that, you know?

That brings me to a, a new thread of conversation we might want to follow in the next episode, which is the question of how the burden of responsible conduct, open practices, who that burden should fall on, and how should we distribute that burden or that responsibility across the members in a team.

[00:19:37] Chris Hartgerink: So join us again in a few weeks and we'll be chatting about this.

Thank you again for listening. Don't forget to join our signal group. If you have ideas around how research or open research or just research in general should develop in the next 10 years, what you would like to see. I think we've had some very concrete points, more confidence building, more clarity and communication and reducing the complexity.

And so we would love to hear from you as well.

[00:20:05] Sarahanne Field: Join the signal team. Give us your feedback.