Monday, August 29, 2016

MP Meetings: David Graham and David Lametti

I've met a few dozen federal MPs so far, and given how important technology law is to me I am always wanting to be introduced to and meet with more.  This is why I was so happy when I heard that a very technology savvy MP was going to be joining us for a GOSLING gathering on April 15th this year.

This was David Graham (Laurentides — Labelle) about the only Canadian MP I've heard being spoken of highly on SlashDot as this is a person already well known in the Linux and FLOSS community.  As technology literacy of politicians is a huge problem (it is what allowed most discussions around the "paracopyright" part of Canada's Copyright law to sound more like a scene from Harry Potter than actual science and technology), I was so happy to be meeting someone who would understand these issues.

When I went to introduce myself, David asked me what I was doing.  Demonstrating what makes him a good politician he explained that he knew me well, and reminded me of the various times we had already met in the past (on a variety of different policies, and even in a social context of when my sister-in-law worked for the Liberal party).  Having a good memory for people is an important part of being a politician, and it is great to see someone who both has the technical know-how as well as those important social skills.

This was something new for me -- meeting with a sitting MP that I met before they were an MP, and that was already actively working on some of the areas of policy that most concern me.

David and his staff invited me to an informal gathering for April 18 to meet with visitors from Sweden's Digitalization Commission. who were in town for a conference. David wouldn't be able to be there himself, but there would be another MP, staff from both MPs offices, as well as Canadian representatives from other parts of the public and private sector.

I was again pleasantly surprised to realize that I had already met Parliamentary Secretary to the Minister of International Trade David Lametti (LaSalle—Émard—Verdun). He had written chapters for In the Public Interest: The Future of Canadian Copyright Law and From "Radical Extremism" to "Balanced Copyright" : Canadian Copyright and the Digital Agenda, and I attended the book launch for (and owned copies of) both books.  Mr. Lametti was already familiar with my particular stance on Paracopyright law.

Both David's are part of a Digital Caucus made up of Liberal MPs and staff.  I hope to meet with them and other caucus members in the future.  There are so many issues to be discussed: while Paracopyright is the issue that dragged me to being so involved with federal politics, the issue this summer is online voting (which if implemented in our current level of digital literacy would become an unaccountable proxy voting system where software vendors would decide the outcome of elections).

Saturday, August 27, 2016

Mauril Bélanger was my reminder of the importance of public service

I spent the morning being reminded of the importance of public service.  I first met Mauril Bélanger in 1996 the context of the Defence Fund for the Montreal Rally (Rally during the Quebec referendum of 1995), and SOS Montfort. When I moved into his riding in 1997 I wrote him a letter.

This was a point of transition in my life.  I wasn't involved in politics until the early 1990's, and at first all I saw in government was corruption.  I was very much at what I now call my "smash the state" phase where I felt the whole lot of current politicians needed to go.   Mauril was someone who gave me a different impression, of someone who even from within government was visibly trying to do the right thing.  While he knew I didn't vote for him or his party, he still listened to me and offered feedback and help. This continued each time we met, where he was always extremely friendly and gave helpful advise on how to move forward on those ares of policy that concerned me the most.

It was with his encouragement and advise that I stepped up in the summer of 2001 and the decade that followed to help defend technology property rights from changes to "copyright" law being proposed.  I have met in person with a couple dozen sitting MPs, hope to meet with more in the future, and this was all largely due to my experience with Mauril.

While I moved to Ottawa-South in 2003, this didn't change how he would greet me the remaining times we spoke with each other.  He wasn't just an MP for the people who voted for him in his riding, but an MP for everyone.  He was a public servant in the most true sense, and I know anyone who knew him will miss him.

Friday, August 12, 2016

Discussion of BDS movement toxic to mainstream politics

As a member of the Green Party of Canada, I am being asked by the party about my views on the recent resolution passed in support of the BSD movement.

Short answer

  • I do not support g16-p006, and believe the resolution should be repealed. I felt embarrassed as a party member to hear that the resolution was tabled and passed.
  • I do not support the motion of the Canadian Parliament that called upon the government to condemn those who promote the movement, and believe the motion should be repealed.  As a Canadian I felt embarrassed to hear that motion was tabled and passed.
I believe Mr. Clement's motion was trolling, and unfortunately some members of the Green Party decided to feed the troll.  This is a no-win scenario for the Green Party and Canadians generally, and the only way to minimize the loss is to exit the conversation quickly.

Longer answer

Having given my own thinking on these and related topics some thought, I came to realize I am part of the Freethought movement.  This goes beyond a support for a strong separation of church and state, which is the issue here, but also to my support for evidence for democracy, open government, and other related movements.

While I support people having their own beliefs, including religious, as part of their own personal lives, I don't believe that this has any place in governance.  Political decisions should be made "on the basis of logic, reason, and empiricism, rather than authority, tradition, revelation, or other dogma" (quote from Wikipedia on Freethought).  I don't believe religious states are legitimate states, and am uncomfortable when dogma (religious or otherwise) is used in decision making in self-called secular countries like Canada (which is more honestly understood as a Christian country that happens to be liberal and tolerant of other religious groups).

Boycott, divestiture, and sanctions are all commonly used techniques to influence a political entity from the outside, and is nearly always understood as a better option than waging war.  It has a long history of being used by both individuals and countries in the west.

The fatal problem with what is now called the "BDS" movement is that it is focused on Israel.  As a freethinker I treat religious states equally, but western society does not.  This special treatment of Israel is most pronounced in North America. There is a strong emotional history when it comes to Israel and the guilt that western (and primarily white Christians) have when it comes to what was done to Jewish people during the second world war.   You can disagree with the emotion, but ignorance of it makes it impossible for you to have political influence.

There is a political technique of equating Israel with the Jewish faith, and equating any critique of either as antisemitism.  While there is no logic or reason to this political technique, it is a highly effective one.  Your logic or reason for being critical of policies of Israel simply don't matter, as the antisemitic label can be thrown at you and you are silenced.  You can disagree with this reality, but ignorance of it makes it impossible for you to have political influence.

I consider the arguments for or against boycotting, divesting, or sanctioning Israel to be largely irrelevant.  Please don't try to "convince" me of your particular position on this topic in the comments. The mere discussion of any criticism of Israel will condemn you in the eyes of mainstream North American politics, and make it impossible for you to have political influence.

No matter what your emotions, the most logical course of action is to leave the hornets nest alone, not beat it with some form of moral indignation and then be surprised when you get stung.

The resolution needs to be repealed and the discussion within the Green Party needs to cease, other than for members to become more aware of why this debate should be left out of party politics.  I left the Green Party in the early 2000's for a while the last time the party (involving both the GPO and the GPC leadership) decided to involve themselves in this politically toxic discussion.  That time it was trying to boycott "David Icke" which had the predictable outcome of promoting this mentally unhealthy person's ideas both within and outside the party, and causing many members to leave the party.

Thursday, August 11, 2016

The Canadiana preservation network and access platform

Canadiana is sending me to Access 2016.  While I expect to learn quite a bit and meet new people, I wanted to make an additional introduction online ahead of time.

I have participated in many changes to the technical platform at Canadiana since I started in January 2011.  As always, what I write is my own thoughts and isn't an official statement from my employer.

Software platform

In early 2011 Canadiana was in transition from an access platform simply called "ECO" for Early Canadiana Online.  This was a mod-perl 1 application that was tightly tied to a legacy version of Apache, was written by an outside consultant, and was in need of upgrading.  I did my best to keep that running as long as possible while keeping our machines secure by having the mod-perl1 application running within a chroot() environment within a newer version of Debian which no longer supported mod-perl1.  This is something we would use linux containers for today, but that wasn't ready in 2011.

The new platform called CAP (Canadiana Access Platform) was also written in Perl, but based on the Catalyst Perl MVC framework, and written by Canadiana staff.  The new platform allowed for multiple "portals" which had their own access theme and content collections (subsets of the full set of AIPs in our repository). CAP used MySQL for most indexes (users and institutions, logging, content metadata) and Solr for search.

Early Canadiana Online remains an important collection, but now shares a platform with other collections.

Between 2012 and 2015 there were major upgrades to the back-end software to allow us to become a CRL certified Trustworthy Digital Repository (TDR).  My part was to upgrade and implement new processes for the automated validation and replication of our file repository.  When I started we were using rsync to copy a comparatively small (A few hundred GB) cmr/ (Canadiana Metadata Repository) directory to a few machines.  The size grew to the point where rsync took a large portion of the day just to realize that no AIPs had been updated. We also ran into a bug in rsync where it can't handle the number of files we were trying to synchronize.

A MySQL based TDR metadata system had been started by a colleague, which I moved to CouchDB to allow the data about our AIPs to be reliably replicated across data centers.  It was now the database that indicted that an AIP needed to be replicated to a specific repository, with rsync only used to reliably transfer all the contents of a specifically identified AIP.  We created processes where a subset of the AIPs would have full md5 checks of their contents, so over time we had validation of every AIP copy that had been replicated to each individual server.

Around this time we also moved from LVM over hardware raid to using ZFS for the TDR file repository, supporting multiple ZFS disk pools individually managed in the TDR file metadata database to optimize storage usage and speed of validation (reduce SAS cabling being a bottleneck).

The new repository system allows us flexibility in how we manage storage, as disk pools can be of any size. A given repository node can be instructed to only store a subset of AIPs as disk space allows, with other nodes having much more storage.  We are not locked in like other systems which only allow storage in a repository up to the size of the smallest node.

In late July 2016 we deployed a major upgrade to CAP to make use of a more Service Oriented Architecture (SOA) for metadata processing, something we call our "metadata bus".  While working closely with the lead developer and our metadata architect, I was the primary on design and implementation of the metadata bus, allowing our lead developer to focus on the upgrades to CAP required to integrate.  While I can write back-end software, I am poor when it comes to front-end and user interface software.

The core of the metadata bus is a series of CouchDB databases, building on the one we created for TDR replication and validation, which we run  microservices against to transform and combine data.  This allows individual microservices to be more easily designed, implemented  and tested than the more monolithic and manual processing we did previously.  This also allows us to easily precompute data which our access platform requires, increasing performance for access.  We continue to use Solr for search, but are now able to harness features we weren't able to with the older revision of the platform.

With this major upgrade fully deployed, I expect smaller incremental upgrades to be deployed more often.


In 2011 all our servers were in Ottawa, with some on Wellington where we work and some at a commercial hosting service on Baseline Road.

During the transition from the "ECO" platform we had 2 servers (primary and backup) for ECO and 2 servers (primary and backup) for CAP.  While at the same co-location company, they were in two different cabinets (I only showed one in image, in Green, as this was the publicly accessible server).

On Wellington we had a workflow server (where we processed images from scanning to ingesting into our file repository), a backup server (Black), and a CMR/TDR server (Red)

In 2012 we moved the servers from Baseline Road to a commercial service in Montreal. We had 2 copies of our image data in Montreal (green for publicly accessible, red for standby), and 2 copies in Ottawa (red for standby, and as part of the backup server marked in black).

In 2014/2015 we added a publicly accessible TDR image data at partner University of Toronto, and an additional working copy of the TDR on Wellington (1 publicly accessible copy, 3 standby copies, and 1 copy part of the backup server).

In 2015 we added University of Alberta as a partner, first shipping a copy of the TDR image data and later in 2016 adding an application server.

We also added an application server and a custom project server to our partner at University of Toronto.

This year we separated the hardware configuration for an application server (which needed RAM, CPU and fast disk for database access and search) and the TDR image data/content server (which needed a large amount of storage, and CPU/RAM for making image derivatives). At the same time we designed and documented a WIP (Work In Progress - marked Orange in Ottawa) server to replace the older workflow concepts, and the concept of a "custom projects" server for special things (Marked blue. In Ottawa that houses development and operations tools such as Redmine, Subversion, and Icinga. In Toronto that houses the CHIN Artefacts Canada linked open data project, the legacy Canadiana Discovery Portal which is no longer part of our platform, the one-off Aboriginal Veterans mini-site, and the Drupal-based corporate site we plan to integrate into our platform)

Most recently we decided to move away from commercial hosting providers, and to rely entirely on partners. We picked up the copy of the TDR file repository that remained in the Montreal commercial provider and it waits in Ottawa as we plan for our next partner joining the preservation network.

If you are at a member institution that would like to join the preservation network, please get in touch. While we are interested to hear from any potential partner, ideal is if we can have more provinces represented in the network.  BANQ?  UVic? Memorial University of Newfoundland?

I just spent some vacation time in Québec, and would love another excuse to visit Université Laval!  :-)

There are some conversations underway about a new partner, and I am excited to see how things move forward.