Theo Presentation

The Authorized_Keys Manager

Michele Azzolari
@macno

GET /api/v1/users/me

{
    "fullname": "Michele Azzolari",
    "description": "Dad, Full Stack Developer, Avid music listener",
    "nickname": "macno",
    "created_at": "1977-04-11T18:40Z",
    "current_job": "CTO and co-founder Fluidware srl",
    "city": "Milan, Italy"
}

GLOSSARY

host: the computer you want to connect to via SSH (where sshd is running)
user: the login name (OS User) you want to connect to via SSH
account: whoever connects, physical person or virtual account (the private key owner)

WHY THEO?

I use several laptops
I manage several servers
I work with different teams

And what happens when...

A new laptop/desktop has to be activated
An old laptop/desktop has to be dismissed
A new server has to be installed
A new member joins the team
A current member leaves the team

You need to update authorized_keys EVERYWHERE!

Of course if you use them

And if you don't... you should!

I hate doing this stuff

Error prone

Files are scattered in several places (a full matrix of hosts/users)

Critical but monkey work

I do the same thing over and over again
If I forget to add a key, someone will complain she can't do her job
If I forget to remove a key, someone will have access to hosts while she shouldn't

You miss the full picture

Did I forget something? There's no way to verify but checking everything
Who can access what?

So?

I definitely want a single place to manage all that stuff!

Then?

Ansible/Puppet/Chef or config management tool of your choice can help or...

Since openssh 6.2 (2013-03-22) a new option is available

AuthorizedKeysCommand

which supports fetching authorized_keys from a command's stdout besides (or instead of) the filesystem.
https://www.openssh.com/txt/release-6.2

EUREKA!

Thanks @johnnyrun for heading me there

The idea

Store accounts/public keys/permissons in a single place and serve them via HTTPS to the hosts

Account connects to host:

ssh a.user@server1

Host fetches authorized_keys from external server by passing its hostname and the user:

GET /authorized_keys/server1/a.user

Server looks up for keys which are authorized for user a.user on host server1:

If it finds any, it returns them to the host:


ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQD[....]GQWv5CcHIe1kQljCzz macno@helsinki.local
ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQ[....]rhAdoKKhbGfAdUw0xv macno@gruff
ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQ[....]347PRq7LplS03D7xUw== macno@habanero

Theo

Involves 3 components

Theo Server
Theo Agent
Theo CLI

Theo Server

Available as docker image theoapp/theo
Uses sqlite3 or MariaDB to store data
Supports caching per-user/host authorized_keys with memcached or redis
Written in node.js

Theo Server

Exposes several REST APIs for manipulating:

Accounts
Groups
Authorized_keys
Permissions

Consumed by Theo CLI

Theo Server

Exposes an endpoint for fetching authorized_keys

Consumed by Theo Agent

Theo Agent

Easy-to-use self-install feature
Keeps a per-user local copy of authorized_keys in the event that Theo Server is unreachable
Can verify authorized keys signatures (and discard them if not valid)
Written in go

We wrote an agent, but it can be replaced by a shell script that curls Theo Server ¹

1 Sigthly harder when signature verification is needed

Theo CLI

Available as npm package theoapp-cli
Supports authorized keys signatures
Written in node.js

What is Authorized Keys Signing?

To avoid unauthorized keys to be returned to SSHD, Theo CLI and Theo Agent both support a way to sign (Theo CLI) and to verify (Theo Agent) each key.

Theo CLI

Using a private key, when it sends a public key to Theo Server, it attaches also a digital signature of the key.

Theo Agent

Using the corresponding public key of the certificate used by Theo CLI, it verifies each retrieved public key with its signature.

Let's put everything together!

To test it locally, you need a machine with node, docker and a virtualization software
In the next slides I will use macos with node 8.9.4, docker 18.06.1-ce and virtualbox 5.2.20 (with vagrant 2.2.1). The VM test-server has been created using this Vagrantfile.

Theo Server

Let's generate some tokens for administration and client authentication

$ head -1 /dev/urandom | base64 | head -3
AWPJQZDWIG8/RgRvJQf7oBOpbt2deEVv/79+O5DTL6wEZC33HsaefORMRhx3ocDQ8X3LsA79MNIU
vW0bhte0SDI4ZtL26m7O4+W3Z44pWfdjHnrZorOtOlvHlqBM6O/UW+dNIDK/GEy8bURlTjpjEHvG
SooEMhppFnYZofmG2Fx438L9i7fM1keXbWQdRWGN+ipmMZ2VP+pQGywNpCIq4K6mdDHfKeLzCEcB

We will use the first string as administration token and the last two as client tokens

Run Theo Server as container

$ docker run --rm -v /tmp/theo:/data \
  --name theo-server \
  -e DATA_PATH=/data/theo.db \
  -e ADMIN_TOKEN="AWPJQZDWIG8/RgRvJQf7oBOpbt2deEVv/79+O5DTL6wEZC33HsaefORMRhx3ocDQ8X3LsA79MNIU" \
  -e CLIENT_TOKENS="vW0bhte0SDI4ZtL26m7O4+W3Z44pWfdjHnrZorOtOlvHlqBM6O/UW+dNIDK/GEy8bURlTjpjEHvG,SooEMhppFnYZofmG2Fx438L9i7fM1keXbWQdRWGN+ipmMZ2VP+pQGywNpCIq4K6mdDHfKeLzCEcB" \
  -p 9100:9100 theoapp/theo


> theo@0.12.0 start /usr/src/app
> node ./build/index.js

theo/0.12.0 listening at http://[::]:9100

That's it!

Theo CLI

Let's install Theo CLI using npm:

$ npm i -g theoapp-cli

Let's set the environment variables:


$ export THEO_URL=http://localhost:9100
$ export THEO_TOKEN="AWPJQZDWIG8/RgRvJQf7oBOpbt2deEVv/79+O5DTL6wEZC33HsaefORMRhx3ocDQ8X3LsA79MNIU"

We can also put these variables in a file and save it as
~/.theo/env
or
/etc/theo/env

Let's create few of accounts


$ theo accounts add \
    --name "Developer 1" \
    --email "dev1@example.com"
+------------------------------------+
{
   "id": 1,
   "name": "Developer 1",
   "email": "dev1@example.com",
   "active": 1,
   "expire_at": 0,
   "public_keys": [],
   "groups": [
      {
         "id": 1,
         "name": "dev1@example.com",
         "active": 1
      }
   ],
   "permissions": []
}
+------------------------------------+
$ theo keys add dev1@example.com \
                    --key "$(cat ~/.ssh/id_rsa.pub)"
+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
{
   "account_id": 1,
   "public_keys": [
      {
         "id": 1,
         "public_key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAAgQD4DaQVgWsexxOabTPxnDau5UsQj5eGUgXLD+28rtWMEAIzehiCcM5y0aOVZO/dBHA7NyHo/jWcw6Dew/Ty/J2WAQwXtV0xFTuA+6TpWBl35UeRJwjuuGnk0nCuvxQL9qs8+7CPis9SNlIQ38wXnsHWuWKEqnTAoO5YdRsjoGtILw== macno@habanero"
      }
   ]
}
+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
$ theo accounts add \
    --name "Developer 2" \
    --email "dev2@example.com"
[...]
$ theo keys add dev2@example.com \
    --key "$(cat ~/.ssh/id_rsa.pub)"
$ theo accounts add \
    --name "Sysop A" \
    --email "sysop.a@example.com"
[...]
$ theo keys add sysop.a@example.com \
    --key "$(cat ~/.ssh/id_rsa.pub)"
[...]

Let's create a group:


$ theo groups add --name developers
+------------------------+
{
   "id": 4,
   "name": "developers",
   "active": 1,
   "accounts": [],
   "permissions": []
}
+------------------------+

Let's add accounts to the group developers:


$ theo groups edit developers --add dev1@example.com dev2@example.com
+----------------+
{
   "status": 201
}
+----------------+

Let's add permission to developers to login as node on host test-server:


$ theo permissions add --group developers --host test-server --user node
+---------------------+
{
   "group_id": 4,
   "permission_id": 1
}
+---------------------+

Let's add permission to sysop.a@example.com to login as admin on host test-server:


$ theo permissions add --account sysop.a@example.com --host test-server --user admin
+---------------------+
{
   "account_id": 3,
   "permission_id": 2
}
+---------------------+

Let's check developers status:


$ theo groups get developers
+-------------------------------------+
{
   "id": 4,
   "name": "developers",
   "active": 1,
   "accounts": [
      {
         "id": 1,
         "name": "Developer 1",
         "email": "dev1@example.com",
         "active": 1
      },
      {
         "id": 2,
         "name": "Developer 2",
         "email": "dev2@example.com",
         "active": 1
      }
   ],
   "permissions": [
      {
         "id": 1,
         "user": "node",
         "host": "test-server",
         "created_at": 1542702951350
      }
   ]
}
+-------------------------------------+

Let's check sysop.a@example.com status:


$ theo accounts get sysop.a@example.com
+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
{
   "id": 3,
   "name": "Sysop A",
   "email": "sysop.a@example.com",
   "active": 1,
   "expire_at": 0,
   "public_keys": [
      {
         "id": 3,
         "public_key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAAgQCtKaAI2AoNoA9gvd0THROZavXoYVa0DoIfrVmBW+FTh8/79yPyf3ZilZVEVpGYJixxneAk1S/k/KsMt68GrG1qaiGLh/pbtT3l6MkHXYqDZI7soi0plUtczXlX8GTyZKML+H/ZOzoltBmV02yUCnSvFecgMvoLzqg1ExGIb1/5AQ== macno@habanero",
         "public_key_sig": null,
         "created_at": 1542702914738
      }
   ],
   "groups": [
      {
         "id": 3,
         "name": "sysop.a@example.com",
         "active": 1
      }
   ],
   "permissions": [
      {
         "id": 2,
         "user": "admin",
         "host": "test-server",
         "created_at": 1542702964294
      }
   ]
}
+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+

Great!

Let's move to Theo Agent

Theo Agent

Connect to the host you want to use (for example a VM running on your host), download Theo agent and make it executable:

vagrant@test-server:~$ sudo curl -L -o /usr/sbin/theo-agent \
    https://github.com/theoapp/theo-agent/releases/download/v0.5.2/theo-agent-linux-amd64
vagrant@test-server:~$ sudo chmod 755 /usr/sbin/theo-agent

Create a Theo Agent user:

vagrant@test-server:~$ sudo useradd --comment 'Theo Agent' \
                    --create-home theo-agent --shell /bin/false

Self-configure Theo Agent and restart sshd:

vagrant@test-server:~$ sudo theo-agent -install -no-interactive \
    -sshd-config \
    -url http://192.168.56.1:9100 \
    -token vW0bhte0SDI4ZtL26m7O4+W3Z44pWfdjHnrZorOtOlvHlqBM6O/UW+dNIDK/GEy8bURlTjpjEHvG
vagrant@test-server:~$ sudo systemctl reload ssh

Let's give Theo Agent a test:

vagrant@test-server:~$ sudo su theo-agent -s /bin/sh -c "/usr/sbin/theo-agent admin"
ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAAgQDOul6NpEnQdRRlt9ALhWjcf1tDkK8VSwtK4FMsDcl8knAr3Ap7oj3/fBhLmlFl9e7AW9J2XFEf6jK+8Sv/N2+vxCMdg6APEsfSa0/gnTnv0JRRmMxBr0kKI/DfrCwGm0075uNmr6JegrLOiGtU/PfmHCgdNJiHqD7Z8301h+XNFQ== macno@habanero

And now from your host... connect!

macno@habanero:~$ ssh admin@test-server
admin@test-server: ~$

CONGRATULATIONS!!!

You made it!

Acknolegments

I would like to thank @gizero for his patience, hints and support.
It's always a pleasure to work with you.

To Tommaso for his beautiful artwork on Theo's logos

Also I want to apologise to my wife and my daughters for the nights and weekends I spent working on this project...
Thanks I love you ❤