Who exports sensitive credentials to shell environment var?

sneakyimp

I've been asked to get acquainted with the SendGrid API -- a system for sending email. Their PHP installation example includes steps for creating a shell script to export one's sensitive API key to a file and then calling source to load those credentials into one's shell environment vars:

echo "export SENDGRID_API_KEY='YOUR_API_KEY'" > sendgrid.env
echo "sendgrid.env" >> .gitignore
source ./sendgrid.env

Their source code examples then retrieve this value using getenv:

$apiKey = getenv('SENDGRID_API_KEY');
$sg = new \SendGrid($apiKey);

Does this seem stupid to anyone else? I can think of a few reasons why I hate this:
export of var into shell environment only lasts as long as your terminal session
this approach doesn't help at all if you are writing a script to be hosted via apache
* sendgrid.env, if stored in one's web root, could be downloaded by attackers to steal your API key for spam purposes

I mean shouldn't one just put the credentials in a PHP file as a constant or $config var and require/include that file? Is there something I'm missing here? I expect a lot of noobs might get really confused by this. It doesn't seem very practical to me at all.

NogDog

I believe the way it's suggested to be done is that the environment variables be set for the apache user (e.g. in it .bashrc file or .profile, or whichever one would apply [not sure, myself]). As that would likely require root access and possibly a reboot[?], the next choice might be to specify Apache environment variables in your httpd config, which would still keep it out of the web-accessible directories, and would only require a httpd restart. Or if that's impractical, then you can do it in .htaccess. https://httpd.apache.org/docs/current/env.html

sneakyimp

Thanks for your thoughtful response.

Yes you are correct that there are various ways to set environment variables to make them available via CLI or apache. I guess my question is why do it this way? What's wrong with putting the credentials in a PHP file outside the web root?.

E.g.:

$sendgrid_api_key = "<ENTER-SENDGRID-CREDENTIALS-HERE>";

And then requiring/including that file? Why introduce those various other approaches which seem way more complicated to me.

NogDog

I think the idea is to keep things like passwords out of the code, and instead in a place where only someone with the applicable login permissions on the server itself could see them. Whether it's warranted in a given situation is a question that probably doesn't have a single, universal answer.

sneakyimp

I'm skeptical that any security is gained by injecting sensitive credentials via an environment variable. Suppose you put the credentials in your apache configuration files. Default permissions on those files is readable by everyone. Anyone on your server could run this script:

<?php
$files = glob("/etc/apache2/sites-enabled/*");
foreach($files as $file) {
        echo file_get_contents($file);
}
echo "\n\n\n";

which shows all the contents of my apache conf files on Ubuntu. The path would probably be different on CentOS/RHEL.

Suppose you put the credentials in the apache user's .profile or .bashrc. In that case, I believe the environment vars would be visible to any PHP script that ever runs on the server.

And supposing you put the vars in one's own .profile or .bashrc file -- this file is also readable by the current user.

One could always lock down the permissions on the apache files I guess, but I just don't think that approach is any more secure and the approach is certainly more complicated. Perhaps I'm missing something.

Weedpacket

True, but if they're able to put such scripts onto your server and run them then it's already game over.

sneakyimp

Weedpacket;11058739 wrote:
True, but if they're able to put such scripts onto your server and run them then it's already game over.

True. My thinking was that on a system with shared hosting folks might get up to something like this. Or some sneak might write a plugin for a popular platform that secretly broadcast such credentials. If someone tricks you into running their code (or is able to run their own code) then it seems to me that the environment variable injection approach provides no security benefit.

Conversely, if no one is able to get code to run on your machine, then the environment variable approach provides no benefit over storing the credentials in a PHP file. Right?

NogDog

I think I read in Essential PHP Security (not sure which box it's buried in) that when you actually put it in the apache user's shell "rc" file, that that file can be read-only by root, which is why he proposed it as a way to provide greater security. But of course, on a shared host, you probably won't be able to do that. But then if you're on a shared host, you probably should not be running anything where something truly bad can happen if someone hacks you. 😉 I think the environment variable stuff probably best works in a virtual (or real) private server situation. If you want to use a "normal" config file approach, you can at least put that file outside of the web root directory tree, to add a little protection.

laserlight

sneakyimp wrote:
I mean shouldn't one just put the credentials in a PHP file as a constant or $config var and require/include that file? Is there something I'm missing here? I expect a lot of noobs might get really confused by this. It doesn't seem very practical to me at all.

You could, but then you would have to keep that PHP file out of version control, e.g., by keeping a template version under version control instead, and then you would need to tell noobs about the template version and how to configure it. In practice, should this configuration file grow to contain other API keys/settings with logic involved, this means that different programmers on the team would be testing the code with possibly critically different configuration files that only they can see. In the environment variables approach, such complex configuration can still go into the configuration file under version control for all to see, with only the values of API keys, passwords, etc, kept out of version control.

Of course, you could try and separate the configuration file from a file that only has the values of API keys, etc, but because it isn't under version control, you depend on programmers not getting lazy and modifying the wrong file in a way that might reasonably be missed in a code review. Perhaps that other file could be in JSON or ini format rather than a PHP file to avoid this, but that could turn out to be just another way to do the environment variables thing, except that you're now loading the "environment" by parsing a data file.

As long as you are not putting credentials under version control and limit access to the production server, I don't think either way provides any real security benefit over the other.

sneakyimp

Thanks for your post.

I would point out that the original post includes this line:

echo "sendgrid.env" >> .gitignore

Seems to me that the need to manage some collection of .bashrc or whatever.env files is no simpler--nay! actually more complicated--than just being careful with your PHP code. Putting template config files into source control is in fact what I've been doing on numerous projects. We've also got some config-loading code that permits config.local files which can be partial to only override some of the values with a local configuration. Personally, I think that using the ENV vars approach looks like a really good way to confuse noobs -- and this is probably a poor decision in a README file for your API library. Especially given that:

laserlight wrote:
As long as you are not putting credentials under version control and limit access to the production server, I don't think either way provides any real security benefit over the other.

Thanks for chiming in!

Derokorian

sneakyimp;11058769 wrote:
Putting template config files into source control is in fact what I've been doing on numerous projects. We've also got some config-loading code that permits config.local files which can be partial to only override some of the values with a local configuration. Personally, I think that using the ENV vars approach looks like a really good way to confuse noobs -- and this is probably a poor decision in a README file for your API library.

This is exactly how I do it, and what I've baked in to my own framework, and thus far no one who I've worked with has been even the slightest bit confused by the wiki for using it: https://bitbucket.org/Derokorian/deroframework/wiki/Configuration

laserlight;11058745 wrote:
As long as you are not putting credentials under version control and limit access to the production server, I don't think either way provides any real security benefit over the other.

I believe this is exactly true. If someone can access your server, it doesn't matter where credentials are, eventually they can find them. If they can't access your server, having the credentials outside of the public folder means they can't see them.

sneakyimp

Derokorian;11058835 wrote:
This is exactly how I do it, and what I've baked in to my own framework, and thus far no one who I've worked with has been even the slightest bit confused by the wiki for using it: https://bitbucket.org/Derokorian/deroframework/wiki/Configuration

I like your approach there and I think we are mostly on the same page inasmuch as we prefer to have our sensitive credentials stored in files in the project directory, rather than in some .bashrc or .profile located god-knows-where-else. Questions about your approach:
is the folder app/config outside the web root? Sure hope so, otherwise x.json might be directly downloaded by snoopy user
perhaps for extra security add DENY rule to apache conf for .json files
Do you exclude your files in app/config from your git repo?

Based on your post, it sounds like you've got all this covered. Just felt like pointing out the obvious I guess.

Still kinda wondering why the sendgrid installation docs describe putting the creds in a shell environment. Kinda makes me think the dev cut their teeth writing some other language or something. Or is this common?

Derokorian

sneakyimp;11058837 wrote:
Questions about your approach:
* is the folder app/config outside the web root? Sure hope so, otherwise x.json might be directly downloaded by snoopy user

I have a separate folder called public at the root of the project structure. This is the only folder that is accessible by web requests.

sneakyimp;11058837 wrote:
perhaps for extra security add DENY rule to apache conf for .json files

I don't believe any such rule is necessary because they are not stored in a web accessible location.

sneakyimp;11058837 wrote:
* Do you exclude your files in app/config from your git repo?

No, I only exclude the config folder. app/config is for git ready files, hence in the example for flickr on that wiki page you'll notice for app/config that api_key and secret are null, and these are then defined per environment in the root config folder.

As an aside, I've been considering changing the folder structure so that all config is in a single place (eg, config/base, config/app, config/local) but haven't figured out a good way to update the FW in an easily migratable way.

Still kinda wondering why the sendgrid installation docs describe putting the creds in a shell environment. Kinda makes me think the dev cut their teeth writing some other language or something. Or is this common?

Environment variables are a very very common for server configurations on windows servers, especially in .NET

sneakyimp

Derokorian;11058839 wrote:
Environment variables are a very very common for server configurations on windows servers, especially in .NET

Auugh! This is precisely the kind of thing I've been secretly and angrily suspecting for some time. Like they've got some wretched .NET evangelist writing the PHP code.

Derokorian

sneakyimp;11058841 wrote:
Auugh! This is precisely the kind of thing I've been secretly and angrily suspecting for some time. Like they've got some wretched .NET evangelist writing the PHP code.

Who says its some evangelist? It might simply be someone with more experience in .NET, just as when I worked in .NET I was much more experienced (and as such more comfortable) with PHP.

sneakyimp

Derokorian;11058843 wrote:
Who says its some evangelist? It might simply be someone with more experience in .NET, just as when I worked in .NET I was much more experienced (and as such more comfortable) with PHP.

I did not mean to offend. I'm just crabby about such things because they seem strange to me. Not the first time I've seen the use of environment vars and getenv. The other time I recall was working with Amazon's AWS tools which I had a terrible time installing. Took forever to get sorted and I vaguely recall that the sensitive credentials ended up in a virtual machine image that was recycled and the credentials had to be changed when those credentials ended up virtual machines used on half a dozen other projects. I suppose there's some lingering resentment.

I'll complain further about PHP frameworks provided by Authorize.net and Paypal. Really poorly written code in my experience. I feel like the boneheads writing them might have been called off some other project to throw a bone to us PHP devs.

End of rant. :queasy:

sneakyimp

So apparently SendGrid gave some advice/justification for putting the credentials elsewhere:

Don’t put your passwords in your code
Use API Keys
Create an environment or credentials file that includes your password(s), but that is in your .gitignore file.
* Put your environment file out of the discoverable file structure - your code can get it, but your web server won’t serve it.

Not very precise language, but "discoverable file structure" suggests their motive for using ENV vars might have been related to this.

Derokorian

sneakyimp;11058911 wrote:
So apparently SendGrid gave some advice/justification for putting the credentials elsewhere:

Not very precise language, but "discoverable file structure" suggests their motive for using ENV vars might have been related to this.

To me this is just saying that the point is that you can't request the file from the web, its still available on the disk if you get into the server. Also, I think it requires more systems knowledge to manage for large scale systems where VMs may be coming up and down all day. (Because now part of bringing up a server is getting the env loaded, instead of just letting the application load it as needed)

sneakyimp

Derokorian;11058939 wrote:
To me this is just saying that the point is that you can't request the file from the web, its still available on the disk if you get into the server. Also, I think it requires more systems knowledge to manage for large scale systems where VMs may be coming up and down all day. (Because now part of bringing up a server is getting the env loaded, instead of just letting the application load it as needed)

I think you and I are interpreting this the same way. Setting up ENV vars on some server is another step that involves files in yet another place, etc. It's easy enough to do once I guess, but when I have to migrate the code base to an upgraded server, it's one more damn thing that needs to also be moved, one more thing to remember, and one more thing to track down in some extra location.