Principles of Open Scholarly Infrastructure evaluation (2022)

Since 2020, twelve organizations signed on to the Principles of Open Scholarly Infrastructure (POSI). It‘s not a meagre list of organizations either: CrossRef, ROR, Europe PMC, and many more.

Today, Liberate Science joins the POSI posse (their words) with our initial evaluation of ResearchEquals. Even though ResearchEquals is an open publishing platform and not open infrastructure per se, we still think it is worthwhile to evaluate ourselves according to these principles.

Our main connotation with this post is that POSI maintains a self-report position. This is great for ease of adoption (no drawn out certification processes), but also means it’s our own and potentially biased assessment. So with everything you read, we would love to ask you: What do you think? We invite you to think critically and let us know. We tried to be strict on ourselves and not give ourselves the benefit of the doubt.

Summary

💚 = good; 💛 = less good; ❤️ = not good

Detailed assessment

Governance

💚 Coverage across the research enterprise – it is increasingly clear that research transcends disciplines, geography, institutions and stakeholders. The infrastructure that supports it needs to do the same.

Any practicing researcher can use the ResearchEquals platform. This ranges from academic researchers to industry researchers, amateur to professional researchers. Any researcher can join and publish, regardless of domain, paradigm, and in a language of their choosing. We recognize that research is pluralist and there’s multiple ways of knowing, which is why ResearchEquals is non-normative in who and what can be published.

💚 Stakeholder Governed – a board-governed organisation drawn from the stakeholder community builds more confidence that the organisation will take decisions driven by community consensus and consideration of different interests.

Our supporting member community convenes quarterly in General Assemblies (started in December 2020). Discussions with the supporting community are held on an ongoing basis on the Supporting member forum. There is no named board, as the entire community shares equal voting rights (1 per member) on decision items.

💛 Non-discriminatory membership – we see the best option as an “opt-in” approach with a principle of non-discrimination where any stakeholder group may express an interest and should be welcome. The process of representation in day to day governance must also be inclusive with governance that reflects the demographics of the membership.

Our supporting memberships are open to everyone, both individuals and organizations. This means anybody can opt-in as long as they pay membership dues - which is considered non-discriminatory membership in cooperatives as well. Technically people or organisations we would not want to affiliate with can become members as well. Note that membership may be rejected as a result of previous code of conduct violations to safeguard the community.

We have room for improvement: We’re currently not openly advertising how to sign up to be a supporting member as we simplify the process to do so. There’s only this outdated original announcement right now.

💚 Transparent operations – achieving trust in the selection of representatives to governance groups will be best achieved through transparent processes and operations in general (within the constraints of privacy laws).

Our supporting members get information about our operations every quarter. They also get the option to petition us for additional information if they wish - similar to freedom of information requests.

As we currently have limited staff, if we are invited or actively participate in other governance groups, the community gets to participate on an ad hoc basis. This means that questions are put to the supporting community for input as they come up.

💚 Cannot lobby – the community, not infrastructure organisations, should collectively drive regulatory change. An infrastructure organisation’s role is to provide a base for others to work on and should depend on its community to support the creation of a legislative environment that affects it.

As a for-profit, we could technically spend money on lobbying. However, we choose not to do so unless with a mandate from our supporting members.

This means that any lobby effort would have to be proposed to a General Assembly, supporting members would have to vote in favor, and only then would that effort start.

If invited to provide the perspective of the organisation, that opportunity will be shared with the supporting member community as well (see also “Transparent operations”).

❤️ Living will – a powerful way to create trust is to publicly describe a plan addressing the condition under which an organisation would be wound down, how this would happen, and how any ongoing assets could be archived and preserved when passed to a successor organisation. Any such organisation would need to honour this same set of principles.

We do not have a living will at the moment as we are still finding our footing for sustainable operations.

During the 2022-2023 timeframe, we will create and implement archival strategy for ResearchEquals content. For the operations, we are continuously developing our business models in line with our manifesto.

❤️ Formal incentives to fulfil mission & wind-down – infrastructures exist for a specific purpose and that purpose can be radically simplified or even rendered unnecessary by technological or social change. If it is possible the organisation (and staff) should have direct incentives to deliver on the mission and wind down.

The goal for ResearchEquals is to shift from a narrative-based model of research to a process-based model of research. As we’re only at the start of this major change, we haven’t yet realistically thought about how to wind this down - it seems premature to do so.

Sustainability

💚 Time-limited funds are used only for time-limited activities – day to day operations should be supported by day to day sustainable revenue sources. Grant dependency for funding operations makes them fragile and more easily distracted from building core infrastructure.

As we are still running on our initial funding from the Shuttleworth Foundation, we are currently running on time-limited funds. In 2022, we’ve grown our own revenue based on consulting work. We project we can run our operations for 2023 based on this consulting income.

We continue our work to generate product revenue from ResearchEquals to further sustain our operations with non-time-limited funds. We aim to move more and more of our revenue from consulting income to product income.

This is an ongoing challenge as we explore business models that provide value to researchers and are in line with our mission of distributing power in research. An example is our “Pay to Close model,” which we’re looking to extend, and other business models we’re currently in the process of developing.

💚 Goal to generate surplus – organisations which define sustainability based merely on recovering costs are brittle and stagnant. It is not enough to merely survive, it has to be able to adapt and change. To weather economic, social and technological volatility, they need financial resources beyond immediate operating costs.

ResearchEquals is run by Liberate Science GmbH - a for-profit organisation. This was an active decision as we want to generate financial sustainability based on our own added value and not be tied to power dynamics with funders (see for example the book "There's no such thing as a free gift"). Of course, this means we are still tied into power dynamics of a capitalist society.

Our aim is to generate profit - and invest that in various ways to further affect and impact change. These investments of our eventual profits can take the shape of cultivating the business, investing in other projects, and allowing our supporting members to decide where to put that profit to collective use.

Our mission is to distribute power, and we‘re setting everything in place to also do that with our profits when they arise.

💚 Goal to create contingency fund to support operations for 12 months – a high priority should be generating a contingency fund that can support a complete, orderly wind down (12 months in most cases). This fund should be separate from those allocated to covering operating risk and investment in development.

Currently we have reserves to run operations for another 12 months, until end of 2023. This is not an explicit contingency fund as such, as we‘re still starting up the business.

💚 Mission-consistent revenue generation – potential revenue sources should be considered for consistency with the organisational mission and not run counter to the aims of the organisation. For instance…

In Liberate Science‘s manifesto and theory of change, we outlined why and how we want to change research. The core of ResearchEquals has to be in line with these values. For instance, we encourage free access to knowledge production. The Pay to Close model was inspired by this value, flipping the „pay to publish“ Open Access model on its head.

Instead of seeing our mission as a limitation for revenue generation, we see it as a catalyst for ideas.

💚 Revenue based on services, not data – data related to the running of the research enterprise should be a community property. Appropriate revenue sources might include value-added services, consulting, API Service Level Agreements or membership fees.

Just nope - we hate surveillance publishing. That‘s exactly why we have supporting membership fees, Pay to Close, and more service based models forthcoming.

💡GHG emissions reporting is our own addition to POSI. We have suggested it to POSI directly - if you like this principle please upvote on GitLab. You can sign in with Twitter and hit the big thumbs up! 👍

💚 GHG emissions are tracked and reported regularly - operating scholarly infrastructures creates direct, indirect, and value chain emissions. Emissions must be transparently accounted for across all three scopes and categories

We reported our emissions for all our completed years in operations. This means we reported on 2019 and 2020 - this was done in our first assessment published last year. More recently, we published our 2021 emissions report. This way we know we've emitted almost 10 metric tons of carbon dioxide equivalent over the organization's lifetime and achieved a 30% reduction from in 2021 compared to 2020.

We currently do emissions reporting on a yearly basis, but want to work towards doing this on an ongoing basis for more sustainable business decisions based on this input.

Insurance

💚 Open source – All software required to run the infrastructure should be available under an open source license. This does not include other software that may be involved with running the organisation.

That’s easy - view our >1,600 commits on GitHub. Fork and let fork! 🍴

💛 Open data (within constraints of privacy laws) – For an infrastructure to be forked it will be necessary to replicate all relevant data. The CC0 waiver is best practice in making data legally available. Privacy and data protection laws will limit the extent to which this is possible

We deposit all metadata for research modules with CrossRef. This includes all citations and abstracts, and whatever other information we can deposit there.

The links between research modules published on ResearchEquals are not made available at scale at this point. We will make that information openly available in the next year through a public API endpoint - thanks POSI for the suggestion!

❤️ Available data (within constraints of privacy laws) – It is not enough that the data be made “open” if there is not a practical way to actually obtain it. Underlying data should be made easily available via periodic data dumps.

Here we clearly need to improve - we don‘t yet have an open API with documentation, nor do we have a data dump. Again a great way to encourage us to do better, POSI! 💕

💚 Patent non-assertion – The organisation should commit to a patent non-assertion covenant. The organisation may obtain patents to protect its own operations, but not use them to prevent the community from replicating the infrastructure.

We do not patent, nor do we want to.

We continuously share our work publicly, creating prior art on an ongoing basis, reducing the chances of us even being able to patent.

If you ever see a patent from us, give us a lot of flack for it please. This should not happen - and we just saved this page to the Wayback Archive as well 😉

Conclusion

Overall, we score 4/7 on governance, 6/6 on sustainability, and 2/4 on insurance. We feel that these scores are only a mere tickle of the work that has to happen to improve. Our sustainability is not safeguarded, despite having a high score on it. Vice versa, we’re proud of our governance process and think this score helps us improve even more. We will be back next year with another assessment.

Thank you for being on this journey with us. If you have thoughts on how we can improve, please feel encouraged to send us an email at [email protected].