Still spinning? The merger of Canadiana.org with the Canadian Research Knowledge Network

(See Part 1, Part 3)

My cubicle on my last day at Canadiana.

If you read earlier articles, you can tell I was excited about the merger possibility when I first heard about it. I looked at CRKN and it had the communications infrastructure that I felt was needed to get out of Canadiana’s DIY/NIH mindset. There were committees to help make key decisions, and there were partnerships with other organizations.

My first meeting with the new CEO, and several other meetings after, indicated exactly what I wanted to hear. What I understood was CRKN’s desire to move the technical team currently providing lower-level services (what I took to mean owning and managing hardware, maintaining custom software, etc) to being involved in cross-sector collaborations (members, other consortia, etc), participating in standards setting organizations, and other activities that were much higher up the technology stack.

The new CEO spoke about overseas trips to participate at standards organizations that I might be interested in. I was very interested, and eager to transition away from DIY/NIH to free up the time to make that possible.

While I was advocating for OpenStack SWIFT and Archivematica at Canadiana, my longer-term hope was that Canadiana (and later CRKN) wouldn’t be trying to duplicate the services of Canadiana and/or CRKN members and partners. I noticed Scholars Portal, the technological service provider for OCUL (one of the 4 Canadian regional library consortia), launched the Ontario Library Research Cloud in 2015 (See OCUL history). 

It seemed obvious to me that, while Canadiana/CRKN needed to create a transition plan, the goal of the plan would be to move these services to Scholars Portal and not continue to manage that duplicate service (Archivematica packaging, large OpenStack Swift clusters, staff training, etc).

I envisioned CRKN coordinating other technological services, possibly with COPPUL offering backup Object storage, and using COPPUL, OCUL, CAUL/CBUA and BCI cloud services for hosting all other services rather than Canadiana/CRKN owning and managing physical hardware in member data centers (Currently Dalhousie University, University of Toronto, University of Alberta, University of Victoria).

During the merger talks there was documentation for the CRKN/Canadiana merger which I followed closely. Some of that documentation became part of a Journal article: “Spinning In”: the merger of Canadiana.org with the Canadian Research Knowledge Network

Excerpt:

Care will need to be taken to ensure that any work supports existing initiatives and players such as CARL, CUCCIO, Confederation of Open Access Repositories, Scholars Portal, and Research Data Canada. It is important to note that Canadiana does not compete with Scholars Portal, but provides complementary capacity focused on documentary heritage content. Given similar preservation models and the ongoing interest in coordinated Canadian digital research infrastructure, there may be emerging opportunities for future collaboration, such as linking data and supporting common TDR nodes for mutual redundant backup and access load balancing.

This specific sentence concerned me: “It is important to note that Canadiana does not compete with Scholars Portal”.

At an administrative level this may appear true, but given OLRC was launched in 2015 and Canadiana was providing a duplicate (if inferior) technological service, that statement wasn’t strictly true.

I wrote the following to Jonathan Bengtson and forwarded to other members of the Canadiana and CRKN boards in summer 2017. I also sent a copy to Clare Appavoo, CRKN’s Executive Director, prior to us meeting for the first time in November 2017.

(Google Docs link)

---

Technological infrastructure: CRKN, Canadiana, OCUL, Scholars Portal

Introduction

As CRKN and Canadiana plan for a merged organization, it is useful to look more closely at the components of Canadiana. While Canadiana is a charity and CRKN is a nonprofit, they exist within a larger context of services offered to overlapping institutions by other nonprofits and consortia. We need to do a competitive landscape analysis to avoid conflict.

I (Russell McOrmond) am the Lead Systems Engineer for Canadiana.org. I am concerned that the relationship between Canadiana’s team of technological infrastructure providers and the technological infrastructure providers for OCUL (Also known as Scholars Portal) is unclear, and that unexpected consequences will result if we don’t create clarity. The merged organization already has plans to help roll out Scholars Portal services across the country.

Teams within Canadiana

From the outside Canadiana might be seen as a single entity, but internally it has a series of departments which have a focus. Understanding these departments is helpful to place them in the larger context.

• Officer: We currently have a single officer, the acting CEO (Previously CIO)
• Production: Currently a team of 5 people work on digitizing and describing (including cataloguing) the resulting images, and managing other processes such as OCR and ingest of that content into Canadiana’s TDR.
• Administration: Currently a team of 2 who handle office management, payroll, and other financial work
• Communications & Partnerships: Currently a team of 1
• DevOps (software DEVelopment, metadata architecture, information technology OPerationS): Currently a team of 3 people that provides the technological infrastructure for Canadiana’s services.

As the Lead Systems Engineer, one of the 3 people in DevOps, I will remain focused on our team.

What does Canadiana’s DevOps team do

Canadiana’s DevOps team researches, creates and/or manages the technological infrastructure used to provide Canadiana’s services.

While we have historically been focused entirely on online publishing the outputs of the production team, a few years ago we started a multi-year project to modernise our platform such that manual intervention by the DevOps team would not be required for most operations. We would be adopting modern platform techniques (Microservices, Docker), open standards (Such as http://iiif.io/ ), and more collaborative development with stakeholders ( https://github.com/c7a ).

The longer term plan was to free up time within the DevOps team to allow us to expand into offering other services for our members. This is services beyond online publishing of scanned/described images.

What is Scholars Portal

Scholars Portal is the technological infrastructure provider for The Ontario Council of University Libraries (OCUL) http://ocul.on.ca/node/135 . Scholars Portal is to OCUL as the Canadiana DevOps team is to Canadiana.

While providing TDR services is a big part of what Canadiana does, and thus what Canadiana’s DevOps team has been focused on, it is a small part of what the Scholars Portal does for OCUL members. Many of the areas of expansion that Canadiana’s DevOps team have contemplated or proposed are already being rolled out by Scholars Portal, including Cloud storage and computing services (OLRC), Geospacial services (Scholars GeoPortal), research data deposit (Dataverse), born digital books and journals.

Clarifying relationship with Scholars Portal

Reading the “Appendix C - Reading Material Merger Background Documents” providing summaries of Canadiana.org and CRKN, you might think Scholars Portal is a Journal TDR. This would be similar to thinking that Canadiana is a microfiche scanner.

Documentation about the merger suggests the new organization doesn’t intend to compete with Scholars Portal. As Scholars Portal is larger than described so far, this will require close attention to the capabilities of both technological infrastructure providers to ensure we aren’t seen by our overlapping membership as offering competing infrastructure. This will be critical as the new organization plans to work with OCUL to offer Scholars Portal services across the country, so will be marketing services of both teams of technological infrastructure providers.

A public directory of Scholars Portal staff http://ocul.on.ca/spstaff lists 28 people, and they appear to be expanding. While not transparent to our members as we have no public staff directory, Canadiana’s DevOps team at one time had 8 people (3 in operations, 2 in software development, 1 metadata architect, one manager, and one coop student). We currently only have 3 people (1 software, 1 metadata architect, 1 operations).

If the multi-year project to modernize Canadiana’s infrastructure is successful, the technology will be much easier to manage. This could free up resources to allow Canadiana to expand into new service offerings, or it could be used as a justification to reduce the size of the team or outsource the management of the technological infrastructure (Including to Scholars Portal itself).

In the “merger considerations and opportunities backgrounder” section of the “Appendix C” document, there is discussion of expansion of Canadiana’s TDR platform. We need to ensure when discussing Scholars Portal that we don’t define their TDR narrowly by discounting their expansion of services, while presuming that any new services that Canadiana offers will be considered part of our TDR.

There are features of the technological infrastructure Canadiana is using to offer our TDR services that may not exist within the infrastructure that Scholars Portal is offering. How these enhancements are offered to our overlapping membership will need to be given adequate consideration. This could involve Canadiana expanding our technology platform to handle new data types, or could be Canadiana working with Scholars Portal’s to enhance their technology platform to have features that make it more trustworthy.

In an ideal scenario the technological infrastructure teams at Canadiana and OCUL would be working closely together to roll out new services to our joint pan-Canadian membership.

Summary

The opportunities described in the “Appendix C” document are all opportunities which a merged CRKN/Canadiana would be well placed to pursue. What is uncertain from the documentation, and thus a concern to the staff providing Canadiana’s technological infrastructure, is what role we will be playing in the future given the potential overlap with Scholars Portal staff and services.