A while back I switched to a Linux based desktop for my work machine as I’d been doing a lot of work in ansible, and having to keep messing with VMs, SSH, and various other hoops was just getting annoying. I’d wanted to experiement for a while anyway. That’s another set of posts. This one is about the 1Password client, and certificate chains.
I’d like to say most folks have a general grasp of browser certificates, the thing that gives a website the little lock icon, but that’d be a lie. A lot of highly technical folks don’t have a general grasp of it, and that’s not to knock anybody. It can get complicated quickly. That’s what I found out this week when the 1Password desktop client refrused to go online. It was essentially stuck in offline mode. Doing all the basic troubleshooting, verified internet connectivity, verified I could get to the 1Password site, checked status websites, etc etc. All seemed good, just not the desktop client.
A little further snooping, and I discovered it was logging the failures to ~/.config/1Password/logs/, with the following entry:
IoError(IoError(error sending request for url (<redacted URL>): error trying to connect: unexpected error: failed to load system root certificates: Could not load PEM file "/usr/lib/ssl/cert.pem"))))
Okay, this was helpful. It was telling us the client couldn’t open the machine’s trusted root authorities list. Checking the file, I found it was actually a symlink to /etc/ssl/certs/ca-certificates.crt (I’m running Ubuntu). Permissions on that file, and the symlink all looked good, I could open the file using vim, head, cat, and any other tool I could use to validate access. Nothing was immediately obvious, even some of the searches across the internet didn’t help much. There was some mentions that on earlier Ubuntu machines, the file didn’t exist, others said they had to create symlinks themselves. But these were all 2017 and earlier posts, and the fact that various tools opened that path suggested those were not related.
So maybe it wasn’t about reading the file, it was about the contents of the file. On looking at the contents of the file, I skimmed over a bunch of certificates, they all looked good, until I got to the very bottom. There was a bunch of random characters, and what looks to be parts of a certificate, but in binary type format. Bits of it looked familiar, they were the root CA for our Windows Certificate Authority. I realized exactly what was going on. I’d copied the DER formatted certificate from our Root CA, not the PEM formatted certificate. When I ran the update-ca-certificates, the script dutifully copied the contents of all the files into /etc/ssl/certs/ca-certificates.crt and created an invalid file, and 1Password couldn’t read it.
The solution in this case was to convert the DER to PEM formatted files using the following:
openssl x509 -inform DER -outform PEM -text -in MyROOT.crt -out MyROOT.cer
Then I removed the crt files, and re-ran the update-ca-certificates command. As soon as this was executed, I clicked the “connect” button in 1Password and it immediately connected.