Look into using one-time-passwords as secondary authentication method for baron

[zooey]

During last BeGeistert, Jonathan Schleifer suggested to use OTP as secondary authentication method on baron, such that people logging in via ssh would have to produce the appropriate one-time-password, too.

While this kind of two-factor-authentication seems to much of a hassle on things like git.haiku-os.org, I think it makes a lot of sense for baron itself (i.e. the hypervisor machine), maybe even for vmdev and vmweb.

One way of implementing this would be to install and configure ​the oath toolkit on whatever server we'd like to experiment with first. The respective SUSE-packages are pam_oath and oath-toolkit, provided by the security-repository.

Of course, for this to work, all admins would need to have some compatible client app running on their smartphone, as otherwise they could no longer log in. One of these apps is FreeOTP, but I think Google Authenticator should work, too.

I have no idea whether to use the time-base (TOTP) or event-based (HOTP) algorithm, so the pros/cons of these require some more research.

This link could be useful: ​http://spod.cx/blog/two-factor-ssh-auth-with-pam_oath-google-authenticator.shtml, this is describing a setup for RHEL/CentOS, but it shouldn't be too difficult to transfer to openSUSE.

[jprostko]

There was actually a talk about this at my local LUG last month, so it's pretty fresh in my mind. Justin (who emailed the haiku-sysadmin list) was also in attendance, so together (assuming he's interested) this may be a good item to tackle, since we got to see a working implementation and the various client apps in action.

I guess something else that could be considered would be using something like a Yubikey for secondary authentication, but in this instance, I admit that using a phone app would make more sense than forcing everybody to buy an additional piece of hardware.

[Centinel]

Yes, the LUG presentation was very helpful - it covered what authentication is (versus authorization), differences between TOTP and HOTP, and then demonstrated a working setup using Fedora and FreeOTP.

I'll contact the presenter and ask him to post his slide deck to the LUG wiki. It could be a valuable learning aid. There is no slide material for the actual demonstration, but I took notes on what he did.

[richienyhus]

Here is a nice list of authentication providers: ​https://twofactorauth.org/providers/

Token2 sounds intresting with the optional ability to recover accounts via txt/SMS messages.

[luroh]

Token2 sounds intresting with the optional ability to recover accounts via txt/SMS messages.

It looks like any random griefer would be able to drain our SMS credits.

[zooey]

Replying to luroh:

Token2 sounds intresting with the optional ability to recover accounts via txt/SMS messages.

It looks like any random griefer would be able to drain our SMS credits.

It does. Additionally, I would rather like to avoid using some hosted service for OTP. From a security perspective (and AFAICS, that's the whole purpose of OTP), it doesn't quite feel right to upload each of our admin's OTP secrets to some cloud. It doesn't matter how much effort the provider's marketing department puts into trying to convey the message that the data is secure: any cloud/server that hosts a considerable amount of secrets is going to be an interesting target for black hats, much more so than an individual smartphone from one of our admins.

[jprostko]

Replying to zooey:

Replying to luroh:

Token2 sounds intresting with the optional ability to recover accounts via txt/SMS messages.

It looks like any random griefer would be able to drain our SMS credits.

It does. Additionally, I would rather like to avoid using some hosted service for OTP. From a security perspective (and AFAICS, that's the whole purpose of OTP), it doesn't quite feel right to upload each of our admin's OTP secrets to some cloud.

I agree. So, do you mind if Centinel and I put together a proof-of-concept on a non-production server, utilizing key authentication backed by two-factor authentication? The plan would be to go with oath-toolkit as mentioned in the original post. Once we get it working for us, there could be some testing done with others before considering pushing it to Baron.

[zooey]

Replying to jprostko:

I agree. So, do you mind if Centinel and I put together a proof-of-concept on a non-production server, utilizing key authentication backed by two-factor authentication? The plan would be to go with oath-toolkit as mentioned in the original post. Once we get it working for us, there could be some testing done with others before considering pushing it to Baron.

No, I don't mind at all - on the contrary: please give it a go :-)

One thing that I forgot to mention in the original description is that OTP can't be activated for all users, as there is for instance a specific user for backup purposes, who can't provide an OTP. So there has to be some mechanism where OTP is activated/deactivated per user (but IIRC the oath-toolkit supports that, so hopefully this isn't going to be a show stopper).

[jprostko]

Replying to zooey:

No, I don't mind at all - on the contrary: please give it a go :-)

Cool, thanks!

One thing that I forgot to mention in the original description is that OTP can't be activated for all users, as there is for instance a specific user for backup purposes, who can't provide an OTP. So there has to be some mechanism where OTP is activated/deactivated per user (but IIRC the oath-toolkit supports that, so hopefully this isn't going to be a show stopper).

Can't we just give the automated backup user it's own cell phone. ;) Seriously though,we should be able to restrict two-factor authentication to users of a certain group(s) fairly easily from what I understand.

I don't currently have access to Baron to tell which groups are being utilized for that user. I can get that information from vmrepo, but if it's any different on Baron, we will need to know about that.

In any case, first we'll try to get it working for all users, and then get it working with a subset of users.

[richienyhus]

Additionally, I would rather like to avoid using some hosted service for OTP. From a security perspective (and AFAICS, that's the whole purpose of OTP), it doesn't quite feel right to upload each of our admin's OTP secrets to some cloud.

There are also self-hosted open source options like ​linotp, ​Gluu, ​tiqr, ​wikid and ​Freeipa.

[Centinel]

I successfully set up TOTP two-factor authentication via oath on jprostko's test server. In my setup, members of the 'otpusers' group are required to present an OTP when logging in via SSH. In the interest of avoid proprietary software (Google Authenticator), I succcessfully used mOTP and FreeOTP to generate OTPs.

I'll wait for him to double-check my work, but if it checks out, hopefully we can move forward with testing.

It's worth noting that two-factor authentication only seems to work with password-based SSH logins; if you are using SSH keys, it will ignore two-factor authentication and log you in directly.

Oh, and for a basic overview of two-factor authentication, here's a link to the slides from that LUG presentation he initially mentioned: ​http://www.wplug.org/mediawiki/images/c/c7/Multi-Factor-Auth.pdf

[jprostko]

Replying to Centinel:

I successfully set up TOTP two-factor authentication via oath on jprostko's test server. In my setup, members of the 'otpusers' group are required to present an OTP when logging in via SSH. In the interest of avoid proprietary software (Google Authenticator), I succcessfully used mOTP and FreeOTP to generate OTPs.

Nice!

I'll wait for him to double-check my work, but if it checks out, hopefully we can move forward with testing.

I'll try to take a look around later tonight.

It's worth noting that two-factor authentication only seems to work with password-based SSH logins; if you are using SSH keys, it will ignore two-factor authentication and log you in directly.

I'm pretty sure that you can define AuthenticationMethods with proper values in sshd_config to get around this limitation, but I could be wrong. I mean, on some of my own servers, I require a public key, followed by the account password on the server, and it works well.

[zooey]

Replying to Centinel:

I successfully set up TOTP two-factor authentication via oath on jprostko's test server. In my setup, members of the 'otpusers' group are required to present an OTP when logging in via SSH. In the interest of avoid proprietary software (Google Authenticator), I succcessfully used mOTP and FreeOTP to generate OTPs.

Great, good work! I personally wouldn't want to use stuff like Google Authenticator either (I use FreeOTP), but I suppose each admin may will whatever they can live with.

I'll wait for him to double-check my work, but if it checks out, hopefully we can move forward with testing.

It's worth noting that two-factor authentication only seems to work with password-based SSH logins; if you are using SSH keys, it will ignore two-factor authentication and log you in directly.

Ouch - we should try to overcome that, as we are using key-based authentication exclusively. Do you think this is a general (i.e. unavoidable) problem or is rather that you have observed that behaviour and there may be ways to fix that?

[Centinel]

It looks like jprostko was right. I followed steps similar to the ones in this article to enable SSH key authentication and OTP to work together:

​http://delyan.me/securing-ssh-with-totp/

Since I didn't want to lock you out of the server, Joe, I commented out the required AuthenticationMethods directive in the sshd config file.

And NOW I'd say we have the test instance where we want it. :)

[jprostko]

Replying to Centinel:

It looks like jprostko was right. I followed steps similar to the ones in this article to enable SSH key authentication and OTP to work together:

​http://delyan.me/securing-ssh-with-totp/

Since I didn't want to lock you out of the server, Joe, I commented out the required AuthenticationMethods directive in the sshd config file.

Thanks. I may have locked you out while I experimented, but I commented it back out myself for now.

And NOW I'd say we have the test instance where we want it. :)

I tested it out, and it ended up being triple factor authentication for me, which needed the public key, the OTP, and then my account password. We are close, but it needs a bit more work yet. I'll see what I can figure out. I suspect it's either something in sshd_config or one of the configuration files you set up.

[Centinel]

I think I understand what's going on.

This is our new configuration directive in sshd_config:

AuthenticationMethods publickey,keyboard-interactive:pam

According to this, a user must have a valid public key and authenticate through PAM in order to log in. SSH keys are easy to set up, so we can ignore that for the time being. Let's look at the sshd PAM file, /etc/pam.d/sshd:

auth        requisite   pam_nologin.so
auth [success=1 default=ignore] pam_access.so accessfile=/etc/security/access-local.conf
auth sufficient pam_oath.so usersfile=/etc/users.oath window=30
auth        include     common-auth
account     requisite   pam_nologin.so
account     include     common-account
password    include     common-password
session     required    pam_loginuid.so
session     include     common-session
session  optional       pam_lastlog.so   silent noupdate showfailed

In the first line, we check to make sure that /etc/nologin isn't present. If it is, authentication fails. ("requisite")

In the second line, if you're set up to use OTP, proceed to the next line. Otherwise, skip the next line.

The third line is what we're interested in. The "sufficient" directive means that if you successfully authenticate with oath, the login immediately succeeds and the entire rest of the file is ignored.

So if you're not set up to use OTP, you process the first line, process the second line, skip the third line, and then proceed with the rest of the file, which asks you for your password. To fix this, we'll have to change the PAM file somehow. I'll give it some thought when I'm at home tonight.

Oh, and don't forget to add an entry for yourself in /etc/users.oath if you want to use OTP on your account. Just copy the first four fields of testboy's entry and change it to point to jprostko. The fourth field is the oath "seed" you'll have to give to the OTP app.

For now, I've left the AuthenticationMethods directive on.

[jprostko]

Centinel, you are right. I was getting the password prompt since I wasn't set up in the system for OTP (other than my group designation). I probably would have caught onto this faster had I dug into the notes you left me more in depth, or if I had simply entered random OTPs and noticed the behavior was the same. So with the current setup, if my SSH key is good, it prompts me for the OTP (I'm using FreeOTP to generate the OTP). If I input the correct OTP, it lets me on the server. If I put the wrong OTP, it prompts me for my password. If that is wrong, I can't get onto the server. If it's correct, I get logged into the server. So yeah, we need to get the PAM file modified to nail that down, then I think this will all be good to go. I'm about to head to sleep, but we can experiment tomorrow.

[zooey]

Centinel, jprostko: I'm impressed :-)

Once you've ironed out that last subtle detail, I think we can copy your OTP implementation onto on of the VMs running on baron (either vmdev or vmweb).

While thinking about the VMs, an "interesting" aspect crossed my mind: vmrepo hosts the git repositories, so a lot of people login via ssh in order to push any changesets upstream. The interesting part is that this includes admins, too. We can't ask every of these users for an OTP every time they push a changeset, so maybe we should limit the OTP requirement to the invocation of sudo? What do you think? Would it maybe even make sense to implement that scheme generally, i.e. only every require OTP for sudo?

[jprostko]

Replying to zooey:

Centinel, jprostko: I'm impressed :-)

Thanks! I'll admit that Centinel has done most of the work so far, although I have been doing testing when I can.

Once you've ironed out that last subtle detail, I think we can copy your OTP implementation onto on of the VMs running on baron (either vmdev or vmweb).

Isn't the plan to update them to OpenSuse 13.2 first due to the "PAM lag" issue present in 13.1?

While thinking about the VMs, an "interesting" aspect crossed my mind: vmrepo hosts the git repositories, so a lot of people login via ssh in order to push any changesets upstream. The interesting part is that this includes admins, too. We can't ask every of these users for an OTP every time they push a changeset, so maybe we should limit the OTP requirement to the invocation of sudo? What do you think? Would it maybe even make sense to implement that scheme generally, i.e. only every require OTP for sudo?

This is an interesting problem. Centinel will probably have a better answer, but I think it should be a matter of adding the pam_oath.so line to the /etc/pam.d/sudo file. It is kind of interesting to only require OTP for running sudo, although that could potentially get annoying if the sudo timeout (timestamp_timeout) isn't set relatively high. I guess it depends on a given admin's workflow though.

[Centinel]

The sshd PAM file has been fixed.

auth        requisite   pam_nologin.so
auth sufficient pam_access.so accessfile=/etc/security/access-local.conf
auth [success=done new_authtok_reqd=done default=die] pam_oath.so usersfile=/etc/users.oath window=30

The first line is unchanged.

The second line is essentially unchanged.

The third line now fails authentication if the OTP is entered incorrectly or succeeds and exits the PAM stack if the OTP is entered correctly. As a result, everything after the third line becomes dead code, but that's okay since it has to do with password-based authentication.

In my and jprostko's initial testing, everything works properly. I think we've finally nailed this. Due to prior commitments, I won't be able to resume work until early next week, but jprostko may finish it off in the mean time.

I'm not a security expert, but I would tend to think that OTP would be more appropriate for logins. After all, it's best to keep people from getting in to begin with. However, I can see why you'd opt for sudo OTP as a convenience compromise.

The PAM stack for sudo is really simple:

auth     include        common-auth
account  include        common-account
password include        common-password
session  include        common-session

Altogether, this (1) prompts you for your password, (2) makes sure that your account and password are valid, (3) checks to see if your password needs changed, and (4) sets up environmental variables. It really couldn't be more straightforward.

So if you wanted to add OTP support, here's how I would do it:

auth [success=1 default=ignore] pam_access.so accessfile=/etc/security/access-local.conf
auth requisite pam_oath.so usersfile=/etc/users.oath window=30
auth     include        common-auth
account  include        common-account
password include        common-password
session  include        common-session

The first line is recycled from one of my earlier sshd PAM stacks: if you're an OTP user, go to the next line, and if not, go to the third line and proceed as usual.

The second line prompts for an OTP. If it isn't entered correctly, authentication fails. If it's entered successfully, go to line three and proceed as usual.

The rest of the lines are carried over from the existing PAM file.

And that's all it should take. Feel free to start from there if you've got time, jprostko.

[Centinel]

OK, I think I've got OTP + sudo figured out.

By default, sudo in OpenSUSE requires you to enter the root user's password. I used visudo and commented out these two directives in order to restore traditional sudo behavior, where the user has to enter his own password:

# Defaults targetpw   # ask for the password of the target user i.e. root
# ALL     ALL=(ALL) ALL   # WARNING! Only use this together with the 'Defaults targetpw'!

If I didn't do that, then PAM would expect you to enter an OTP based on the root user's account.

Next, I had to change the /etc/security/access-local.conf file that PAM references when determining whether or not to require an OTP. Here's what it originally said:

+ : ALL : LOCAL
- : (otpusers) : ALL 
+ : ALL : ALL 

The first line indicates that "local" users (for our purposes, everyone who is signed in over SSH) are not bound by OTP. That's why it didn't work when jprostko attempted to transplant my sshd PAM stack directives into the sudo PAM stack - you were already signed in, so it triggered the local user exemption.

The second line requires OTP for accounts in the otpusers group.

I deleted the local user exemption:

- : (otpusers) : ALL 
+ : ALL : ALL 

Now, 'otpusers' accounts are immediately bound by OTP, regardless of where they're logged in, and everyone else is exempt. Nice and simple.

Here's my sudo PAM stack:

auth [success=1 default=ignore] pam_access.so accessfile=/etc/security/access-local.conf
auth requisite pam_oath.so usersfile=/etc/users.oath window=30
auth     include                    common-auth
account  include                    common-account
password include                    common-password
session  include                    common-sessio

If you're not bound by OTP, you skip the second line; otherwise, you have to enter an OTP.

I'll let jprostko verify this, but everything seems to be in order.

[jprostko]

I can confirm that Centinel's solution works. Currently on the test deployment (at least for my account), I need a OTP to get on the server and then an OTP to use sudo. Keep in mind that if you log in and then immediately go to use sudo, you have to wait for the next OTP in the rotation. This makes sense though, since they are one-time passwords and all.

Do we need su -l to require a OTP as well? I would think we'd hit the same problem that Centinel hit when he made the change to having sudo use the user's password and not the root password for the server. I kind of think having OTP for sudo is sufficient.

In any case, Oliver, how would you like to proceed? Should only sudo utilize OTP, or would you like the server login to require it as well. Maybe it depends on the server?

[zooey]

What do you think about this scheme:

• on baron, vmdev and vmweb:
  • require OTP during login via ssh
  • do not require OTP for sudo
  • do not require OTP for su -l

• on vmrepo:
  • do not require OTP during login via ssh
  • require OTP for sudo
  • do not require OTP for su -l

The difference in behaviour could be a bit awkward at first, but I think one gets used to it. I'm not sure about the security impact of this, so if anyone thinks this is a bad idea, please speak up.

[Centinel]

Since sudo provides a subset of su -l's functionality, it seems ineffective to limit a part and not the whole. Then again, I'm not a security expert, so take that as a layman's opinion. It's really a matter of security versus convenience.

If you wanted to focus on security, you could leave OpenSUSE's default sudo behavior in place and create an OTP seed specific to the root user. That way, users who wanted to sudo or su -l would be required to enter an OTP specific to the root account. It obviously wouldn't make sudo as secure as requiring user passwords and user-specific OTPs, but it would plug the security hole coming from su -l unprotected.

On the other hand, I am compelled to admit that restricting sudo is better than nothing.

[zooey]

Centinel: many thanks for the input. I think you're right, it just makes sense to use OTP for su, too (I personally never use su anyway, so that won't cause any inconvenience for me ;-)

[zooey]

While OTP-support has been implemented on vmweb, the requirement to use PAM has indeed brought back the strange phenomenon that the first login of some period (a day, IIRC) will take ages (around 1 minute). So far, the logs haven't shown much info as to what exactly causes the timeout. All I know so far is that the time seems to be spent in the PAM stack.

Activating debug output in PAM didn't yield any more relevant info either.

[nielx]

The haiku-sysadmin user no longer exists, changing to haiku-web.