The Tao of the Registry

sneakyimp

Apologies for a long rumination post. I'm trying to move rotten old code to a shiny new project and hope to eliminate the pain I've been dealing with. It's also a fresh new year and a good time to sharpen the skills...

I have a lot of configuration values that need to be accessible throughout my project:
Cloud API credentials (CDN user name & key, etc.)
Payment gateway credentials (Paypal, Authorize.net, etc.)
Mail credentials (Sendgrid or MailChimp, etc.)
Inumerable site-specific configuration settings (e.g., CURRENT_SPECIAL_OFFER_SRC_FILE)

Note that these are above and beyond your usual site configuration settings (db credentials, file system locations, domain information, etc.).

My question is this: How should one deal with all these various types of configuration such that they are easily accessed in code? This may seem like a simple question, but I'm not really looking for a simple answer. I have revisited old posts here on phpbuilder.com, I have checked out CodeIgniter's Config Class, and I've been looking at a couple of books that both mention Martin Fowler's discussion of the Registry pattern. Anecdotal experiences and solutions are welcome.

One of the biggest questions I have is whether a registry class should be global/canonical/static across my entire project. This would be both good and bad.
GOOD: One registry we can always locate and retrieve the authoritative value from. The one, true registry would eliminate a lot of confusion in a project.
BAD: Global objects are vulnerable to name collisions and tend to undermine the modularity of one's code, reducing its reusability and ability to place nice with other code. Global/static/shared variables can lead to race conditions or deadlock or other nastiness in multithreaded/concurrent execution scenarios. (Other?)

Another giant question is that of persistence mechanism. I.e., should I use config files or a database to specify config values?
DB PROS: easy to set up a 'settings dashboard' where an admin can alter these settings via some web interface. Writing a single new value is easily accomplished by writing one or more records. Easy to retrieve values via lazy loading.
DB CONS: Requires db connectivity, dependent on db connection for operation which would essentially preclude settings for any operations that should proceed in the absence of DB connectivity (um LOGGING cough cough), specification of arrays or objects or other '3-d' data structures requires trickery or some kind.
CONFIG FILE PROS: can lead to convenient organization by separating settings into distinct config files or otherwise utilizing fileystem as means of dividing namespace via folders-and-files (think Windows Registry). works even if DB connection fails completely. Easy to define '3-d' data objects like arrays and objects and so on.
CONFIG FILE CONS: difficult to change a single value as it'd be tricky to rewrite the file. no lazy loading really as you kinda have to parse the entire config file. performance? caching, etc.?

Also, it should be pretty apparent that some of these values (payment gateway credentials, Amazon EC2 credentials, mail credentials) are sensitive information. Security is a great concern here. Some have suggested that such credentials are encrypted. It makes me wonder at what point they get encrypted and at what point they get decrypted and which ones get encrypted and which don't and of course would this even make sense because the encryption key would have to live on the server anyway.

There's also the question that my code may end up being tightly coupled to this Registry concept. What is a reasonable way to limit one's exposure to this tight coupling issue without jumping through a zillion hoops? I'm inclined to think in an MVC project that limiting Registry access to my Controller is one way, but I worry that might be extreme. Hard to say until I'm deeper in. Perhaps models might also access it? sigh

There's a fairly interesting example here that contains some snarky comments and also links to discussions about Registry vs. Dependency Injection.

All discussion and advice is welcome.

NogDog

For settings that need to available everywhere in your project, maybe you could use $_ENV and set them in the actual environment for the user that actually run the PHP scripts (e.g. "apache" or "nobody"). Efficient/effective for an app you have full control over -- which it sounds like this if for? -- versus something to be generally available to the masses.

sneakyimp

NogDog;11044619 wrote:
For settings that need to available everywhere in your project, maybe you could use $_ENV and set them in the actual environment for the user that actually run the PHP scripts (e.g. "apache" or "nobody"). Efficient/effective for an app you have full control over -- which it sounds like this if for? -- versus something to be generally available to the masses.

Nearly all of my projects in the past few years have involved one domain being hosted on a single server and I usually have root/admin access, but rarely "full control." I fantasize about full control.

That said, I'm opposed to putting SetEnv in the .htaccess file. I've seen .htaccess files get really ugly with mod_rewrite rules and I've seen .htaccess files moved to a proper apache conf for performance reasons. I really do like the idea that each dev's workstation can set its own $_ENV values and these need not find their way into the git repo, but generally dislike the need to leave PHP to establish settings to be used in PHP. It's one more thing a dev/client needs to understand and know and one more thing that might need to be migrated from server to server and it may require reboot of the server etc. etc.

Or do you mean simply to create a PHP config file that simply adds values to $_ENV?

NogDog

I was thinking along the lines of storing global, static config values in the user's OS environment variables, which should then be available via $_ENV. I've never really messed with it, but saw it suggested as a way to store DB passwords -- I think because the shell config file (.bashrc?) can actually be set up with permissions so that only root can read it? Just an interesting idea I thought I'd throw out there.

sneakyimp

I have issues with $_ENV:

1) I can't seem to get any values into it.

For apache, I tried this in my .htaccess file

SetEnv SOME_VAR some-value

This does find its way into PHP if I use getenv:

var_dump(getenv("SOME_VAR"));

will yield this:

string(10) "some-value"

however, $ENV is still completely empty:

var_dump($_ENV);

only results in:

array(0) {
}

For CLI, I can see from the linux printenv command that I have all kinds of environment variables defined. This PHP:

var_dump(getenv("HOME"));

yields this result:

string(15) "/home/sneakyimp"

However, this PHP:

var_dump($_ENV);

Still shows $_ENV to be empty:

array(0) {
}

This may be specific to my OS/installation particulars, but that still presents an obstacle. The point is to come up with a viable Registry concept that works across all platforms when you need it to without extra difficulty.

2) This sounds like a good way to expose sensitive data. Looking at the output of my printenv command, I see that I have a variety of old Amazon EC2 credentials in my .bashrc file which are available from any PHP script that gets executed via command line on my workstation:

EC2_KEYPAIR=MyKeysSneakyimp
EC2_URL=https://ec2.us-east-1.amazonaws.com
EC2_PRIVATE_KEY=/home/sneakyimp/.ec2/pk-A96FBLABLAHBLAH.pem
EC2_CERT=/home/jaith/.ec2/cert-a96fBLABLAHBLAH.pem

There is of course a similar threat if such values are exposed through an Apache SetEnv directive. Installing sensitive data such that it's available to all scripts that run on a given machine sounds like a good way to invite exploitation. Your credentials maybe less likely to end up in your github repo but you might still be attacked via code injection or unsafe 3rd party code.

other problems:
Injection of ENV values via .htaccess or .bashrc makes it difficult to define '3d' data objects which could easily be attached to a PHP Registry object
Requires additional configuration effort during initial setup, when copying source to other dev's machines, when migrating from one machine to another. Basically, it's yet another moving part.
no permanent changes to values stored this way can be effected at all from PHP
if we have to use .htaccess to set these vars and check the file into git repo, devs could end up over-writing each other's values.

pros:
works without any database at all
available globally in any script at any scope
* values are read-only and safe from being overwritten without some fancy/tricky PHP

NogDog

Just to continue brainstorming, (i.e. not saying this is good or bad, just thinking about it), you could set up an auto_prepend_file that would create a system-level registry object, using a namespace to make it distinct from any other object of that name.

sneakyimp

Here are some interesting Registry options.

This is a Settings class for CodeIgniter. It comes with a particular db table structure. You can define key-value pairs in your database and you can autoload them, define groups, edit/change values, etc. It looks pretty straightforward and handy. It looks like one can make multiple instances of this registry if one needs to, but I'm not sure there's any way to separate out the values one registry instance might load from the database table versus what some other instance might load. There does seem to be some way to divide up settings into option groups but you are still at risk for name collisions between distinct registry objects because they share a db table.

Derokorian's Config class, specific to his own framework. Looks like it encodes config in flat files as JSON-encoded text. Might be fairly easy to alter this config and export back to a new json-encoded config file. This registry works on a common principle I've seen: the registry uses a static array var to hold the settings. In this implementation, all methods are static as well. This results in a functionality similar to just using $GLOBALS["key"] = $some_value except that it has handy functionality to load from configuration files (but none to write config files apparently). The difference is that this one might avoid name collisions with other code manipulating $GLOBALS by defining a static variable array Config::$config.

sneakyimp

NogDog;11044639 wrote:
Just to continue brainstorming, (i.e. not saying this is good or bad, just thinking about it), you could set up an auto_prepend_file that would create a system-level registry object, using a namespace to make it distinct from any other object of that name.

Thanks for the brainstorming! As I hinted in titling this post, I am in fact hoping to keep my mind open to a possible better way for dealing with Config/Settings/Registry. In using auto_prepend_file, we are also talking about changes to PHP.ini which is also tricky (out of bounds!) for inexperienced developers. As one must change PHP.ini (or .htaccess), it adds one more damn thing that must be done when setting up a new workstation. Either that or one more file you must exclude from the git repo. I think most frameworks worth their salt should provide hooks where one can define their own classes or data objects early in the framework's init operations. I know CodeIgniter provides a pre-system hook (see hooks).

There are some things I like in a registry:
Make it quick and easy to get access to settings/config/registry at any scope of code
Plays nice with anyone else's framework without risk of name collisions (maybe we should use a namespace?)
Sometimes it might be useful to have more than one registry??
Can hold more than just scalars (like arrays and objects too)
It's really easy to organize config values. Not sure if this means a database or flat files. Parsing and/or writing flat files seems more of a chore, but Derokorian's approach looks pretty interesting.
Supports key paths or some other means of grouping (and separating) registry values. I.e., 'namespacing' or 'registry folders'. This also adds flexibility to avoid name collisions.

sneakyimp

This is a pretty good article on creating a PHP registry object. It points out one of the main big advantages, namely that it can make it very easy to make sure you create one and only one database connection throughout your code. From the article:

This is a better method. Why is this better? After all, it has more code! That’s right, but it’s better in several ways. First, it allows us to connect to more than one database server. This is critical, especially for large applications where your data may be assembled in a master-slave setup. Second, it allows you to store not just one object, but an unlimited number of them, and access them all at the same time. We have essentially created a core repository for objects, all of which can be accessed by any other object by simply requesting it.
...
There are some drawbacks, though. First, the Registry class throws an exception if you request an object that doesn’t actually exist. This means that planning when your objects will exist is critical. Second, we are to some degree hard-coding in an object reference, and this can mean that changing it can be tough.
...
There’s also a great amount of debate over whether to use the registry pattern or dependency injection. Much of this argument focuses on access control and concurrency issues, and source code readability.

The comments on this page also seem pretty astute:

Yes, congratulations on creating a global scope which throws exceptions… 😉

There's also a lot of talk about how this kind of Registry/Singleton pattern makes testing difficult, but I don't understand why.

NogDog

I think it depends on how you use the Registry as to its impact on testing (at least to some degree). If classes that use it compose it from within (e.g. call Registry::getInstance() from within their constructor), then you have a strong dependency that is not easily worked around; whereas if you pass the Registry object into the other classes via method arguments, then you can pass in a stubbed or modified version (child class or implementer of same interface?) of it to use in a test.

sneakyimp

NogDog;11044655 wrote:
I think it depends on how you use the Registry as to its impact on testing (at least to some degree). If classes that use it compose it from within (e.g. call Registry::getInstance() from within their constructor), then you have a strong dependency that is not easily worked around; whereas if you pass the Registry object into the other classes via method arguments, then you can pass in a stubbed or modified version (child class or implementer of same interface?) of it to use in a test.

This is a bit over my head I think. Or rather I have a vague recollection that it relates to Unit Testing.

I'm aware that any classes directly instantiating/fetching a Registry instance will be tightly coupled to that Class and its method of retrieval. If you use something like Registry::getInstance() then you are automatically assuming a few things about the current (possibly global) namespace. Namely that there is a class called Registry and it has a getInstance static method, and probably other things, etc. I've seen a lot of behavior in Joomla like this with the JFactory class.

I can vaguely appreciate that passing a registry object in lets you possibly generate a stub/fake one for a quick test but I think I fail to see why this is especially important. I have a vague recollection of someone saying that Joomla's JFactory class made unit testing very difficult. I'm still not sure why that would be.

NogDog

That's essentially it: if you build a unit test to test Foo, and it creates an instance of Registry internally, then that unit test will always fail if there is something wrong with Registry, it doesn't exist, etc. If, however, you pass it into the Foo class via its constructor's args (or other method's), then you can, at the very least, pass in some test version with the bare minimum of properties/methods that you need for the class being tested -- even having the unit test build its own, purpose-driven version of that Registry class if needed (otherwise just making sure its definition is loaded before instantiating the class under test).

Or something like that -- I don't do near as much with unit tests as I should. 🙂

Derokorian

The idea behind unit testing is to reduce the number of items under test when the test runs. This helps to ensure that if/when a test fails, it is really because of the code you believe it to test and minimizes where to look for mistakes. The reason dependency injection makes this easier, is because no matter how much your class relies on outside code, you can still test just the class itself.

For example, take this test of a controller. While it relies on a model existing, and behaving as intended. The tests do not rely on the actual model to exist.

class UserControllerTest extends PHPUnit_Framework_TestCase
{
    /** @var  App\Controller\UserController */
    private $oController;

/** @var App\Model\UserModel */
private $oModel;

public function setUp()
{
/**
this is where we create a mock of the model this controller relies on, defining the methods we expect to exist
  we also have to pass in a mock of the database connection class to the model, as it relies on that
  and phpunit mocks objects based on an existing class if it exists (however if it doesn't because it wasn't loaded
  this mock continues to function properly for the controller under test
*/
        $this->oModel = $this->getMock(
            'App\Model\UserModel',
            ['validate','getUser','insertUser','updateUser','checkLogin'],
            [$this->getMock(
                'Dero\Data\PDOMysql',
                null,
                ['default']
            )]
        );
/**
Here we inject our mocked model into the controller under test, limiting the tests
  exposure to just the controller's code and nothing else
*/
        $this->oController = new \App\Controller\UserController(
            $this->oModel
        );
    }

public function testLoginSuccess()
{
    $_POST['username'] = 'user';
    $_POST['password'] = 'pass';

    $oLoginRetval = new Retval();
    $oLoginRetval->Set(123);
/**
This is where we mock the return value of a function on the model
  that the login function on the controller relies upon. We want check
  login to return a Retval that we created just above
*/
        $this->oModel->method('checkLogin')->willReturn($oLoginRetval);

    $oGetRetval = new Retval();
    $oGetRetval->Set([['user_id' => 123]]);
/**
Again we mock another function on the model that the function under
  test relies upon.
*/
        $this->oModel->method('getUser')->willReturn($oGetRetval);
        $mRet = $this->oController->login();

    $this->assertNotEmpty($mRet);
    $this->assertArrayHasKey('success', $mRet);
    $this->assertTrue($mRet['success']);
}
}

The biggest benefit tests like this provide is learned when using a continuous integration system like Jenkins or Travis (why they are male first names, I have no idea). My little framework uses TravisCI which is free for public repos on GitHub. Now my little framework has only meager tests at the moment, but they get run on each new patchset. You can see here the output of the latest successful build. When a unit test fails, the build fails and I get immediate notifications. Not only does it tell me exactly which unit test fails, but because of the limited exposure of each individual test I can narrow down the problem code to within a few lines.

If it were not for dependency injection, then you depend on many classes being correctly defined and included in order for a test to run. Which may cause confusion when trying to identify the cause of the failure.

PS. Sorry its not on topic of the original discussion.

sneakyimp

This is all great stuff, guys. Thank you.

So, I realized that it is my natural tendency to follow a depency injection approach for the most part rather than a service locator approach (which is what registries are reallly about I think). I've seen many cases where someone uses a service locator (like JFactory in Joomla) deep inside some class somewhere to get a handle on the global database and it has always bothered me mostly because I would wonder "what if I don't want that database but a different one instead?" It always seemed like a good idea to specify which database a class should be using.

The ease/difficulty of Unit Testing as it relates to this is also of concern. I don't myself do much of it at all, but I want to get more in the habit of doing so.

On the other hand, suppose a class has some elaborate behavior that works with many configuration values, a database, a couple of objects, etc. etc. etc. It seems really awkward, tiresome, and error-prone to have to pass in ALL of these things down through some series of function calls or class constructors to where the work actually gets done. ESPECIALLY if these values are simply number or string constants. Like suppose deep in some class hierarchy I encounter a situation where I need to send an email which has to go through sendgrid or something because my server is a VPS and therefore banned from sending email as policy. Do I really need to propagate an email host, username and password down through this monkey chain of function calls and constructors so that my deeply nested function can just send its godforsaken email?

Seems like we have a few options:
1) Yes, propagate all the consts, vars, services, everything down through the monkey chain (BAD)
2) No, use a service locator for everything (also BAD as everything is all classes are tightly coupled to some rigid structure)
3) Make judicious use of each approach as needed.

The question almost seems to be at this point WHICH information should be found using a service locator and which should be passed in via dependency injection?

I'm starting to think that a controller should have access to a Registry or service locator and be responsible for passing dependencies in to all subordinate classes...or something like that. Things seem more complicated.

Perhaps it would make sense to talk about which types of info should be in a registry instead? I've always been suspicious of putting a complicated DB object in a registry. Certain global constants, on the other hand....

NogDog

Of course, if your Registry object has everything that classes in that call chain need, at least it's the only parameter you have to pass along. 🙂

sneakyimp

NogDog;11044687 wrote:
Of course, if your Registry object has everything that classes in that call chain need, at least it's the only parameter you have to pass along. 🙂

Great point, which just caused me to have something of an epiphany about CodeIgniter. If I'm not mistaken, the database object established in one's CI config is not globally available except via the CodeIgniter singleton. This CI singleton is essentially a 'spine' or 'hub' of the entire CI application and one's db, config, etc. is all attached to it. Curiously, CI's design seems to pains seem to make sure its properties are available in all of one's controllers, models, etc. even though you can always grab the CI singleton via the global get_instance() function. I can't quite figure out why CI tries to attach this complex singleton to everything except to make its properties conveniently accessible throughout the application.

Seems to me there might be some advantage to NOT making a registry accessible via singleton and passing it as a parameter, namely that you don't preclude the possibility of some other such registry existing. Conversely, the advantage of using a singleton service locator object is that you don't have to keep passing this parameter back and forth. As for passing one of these complicated registry objects as a parameter, it still seems to me like your code is going to be tightly coupled to its structure.

Man my head is kinda spinning. Tradeoffs, tradeoffs.