Ticket open with a header() problem --- any reason I can't close this?

[dalecosp]

I have this bug report, regarding header()s that were found in an app without an exit() call following them:

Ticket 2265 wrote:

In looking at login redirection today (filename.php, second incidence of "case 3"), I noted that <<Package_redacted>> was written with many Location header() calls, but none of them were followed by an exit.

This was actually the cause of a problem I was having (I think), with redirecting a logged-in user back to whence they came.

Not calling "exit" after sending a location header can leave the server to execute more code and increase server load.

We should add an "exit" call to most location headers unless I can be shown a reason otherwise. I should probably examine their "app-bottom.php" to see what else they might be doing.

In looking at this, I see no reason not to change "most" to "all", simply add the exit() calls and resolve the ticket. App-bottom.php is included at the very end of the script, after much more code .... it seems very unlikely that app-bottom.php would/should(!) ever run.

Can you think of any reason that I shouldn't just add exits and resolve this bug?

[NogDog]

One could certainly intentionally and correctly (if possibly confusingly) set a location header at one point in the code while still wanting to execute more code. Admittedly, I don't recall ever doing so (though I certainly have with other types of headers). Therefore, I might at least do a directory search for "header(" and see what the scale of the issue is. Or, if you have a nice, full, and robust automated regression test suite, you could just change them all, then run the test suite and see if anything breaks. What: you don't have one, either? 😉

[Derokorian]

I wouldn't just blindly add it. We purposely don't exit after to do reporting and logging data after responding to the user with the call. This may seem like a minimal improvement but it saves time for our servers and the user.

[dalecosp]

NogDog;11048127 wrote:

What: you don't have one, either? 😉

LOL, the project doesn't even have a sandbox. Fortunately(?) for me, I can't say that I was responsible for the original implementation, only the myriad improvements 😉 😉 😉

Derekorian, that's why it's not been closed yet ... wanted to try and figure out if anything from app-bottom was actually being run. Can you clarify this, though? What "minimal improvement" and how does it save time?

Derokorian wrote:

This may seem like a minimal improvement but it saves time for our servers and the user.

[Derokorian]

Apache is no longer holding on to the PHP thread and can instead respond to another request, at lesat the way our server is configured it happens this way. Additionally for the user, instead of waiting for file and/or database IO they immediately get their redirection and can request the new URL. Again, its super minimal but we're about saving milliseconds here as much as possible to improve user experience.

[dalecosp]

Derokorian;11048165 wrote:

Apache is no longer holding on to the PHP thread and can instead respond to another request, at lesat the way our server is configured it happens this way. Additionally for the user, instead of waiting for file and/or database IO they immediately get their redirection and can request the new URL. Again, its super minimal but we're about saving milliseconds here as much as possible to improve user experience.

So you issue header(), run a logging function and then exit()? I thought exit() was called to terminate the PHP process (thus releasing the Apache worker, as you imply --- but your comment seems to indicate the exact opposite of what I was thinking ...)?

[Derokorian]

Exit ends the php process, there are no exits in our code. Apache can be configured to close the connection when it receives everything (this can cause issues with streaming if our content length header is smaller than the actual content). I'm not sure the exact configurations, but it can be tested easily. I can set a 10s sleep after the connection should end, and I can see the browser finish the request in normal time, but php still exists to do things. I wish I knew what about the configuration caused that, but I can't find anything (and this was all configured long before I started). For redirects we send a 30x header + location and these two together are enough to finish the request, so I think having both of them causes apache to "flush" the response..

I'm sorry I'm not actually being helpful, just describing what I see here. I'd also point out, my personal projects don't work this way, so its something different from our corporate system vs my system.

But to answer your statement, YES exit terminates the PHP process. However, PHP and apache are separate processes and there's nothing that I know of that says "processes apache forks (aka php) to handle a request must be completed before apache can complete the request".

[dalecosp]

Thanks for the explanation. I guess one thing I should never assume, and often do, is that apples = oranges for any two given systems --- I know, for example, that's not even true for our properties, as some run PHP as CGI under suPHP and others use PHP as an Apache Module. For all my "grok the command line", I'm not expert at all at system admin, low-level networking, calculus 😉 😉

Maybe I'll file a ticket with the hosting company. We've not bothered them for a while - one of their guys can scratch his head a bit on this one 😃

[dalecosp]

Experimented with this:

<?php

//header("302: Found");

header("Location: $some_other_valid_URL");

file_put_contents("this is post header","/tmp/postheader.txt");

Nothing gets written, whether or not the 302 is sent first. It seems likely, though not definite(?), that nothing is happening after any header() call for us, so this may be a non-issue here.

[Derokorian]

well you know why nothing was writting in /tmp right? file_put_contents(filepath, data).

This worked for me (with and without the 302):

<?php

//header('HTTP1/1 302 Found');
header('Location: http://google.com');
file_put_contents('/tmp/test.log', $_SERVER['REQUEST_URI']."\n", FILE_APPEND);

[dalecosp]

And this IS why we hate PHP sometimes, right? 😉 😉

I mean, I love teh manual; I just hate looking up the order of args for every function I use that has more than one in standard use .... :rolleyes:

[Bonesnap]

dalecosp;11048267 wrote:

And this IS why we hate PHP sometimes, right? 😉 😉

I mean, I love teh manual; I just hate looking up the order of args for every function I use that has more than one in standard use .... :rolleyes:

Doesn't your IDE tell you? 🙂

[NogDog]

Bonesnap;11048295 wrote:

Doesn't your IDE tell you? 🙂

Mine does, as well as for any other functions/methods I've defined. 🙂

[dalecosp]

Of course, you assume that every project or file-to-edit gets loaded into an IDE.

Some of them here won't fit. Probably should look into dumping NetBeans, as it's apparently a RAM hog, too. How would PHPStorm handle a live remote project with 100K files?

[NogDog]

100,000 files of code for one application? Please tell me you meant something else -- otherwise put your keyboard down, stand up carefully, turn around 180 degrees, and run as if your life depended on it.

All I can tell you so far is that PHPStorm works fine with our growing Laravel-frameworked application with some hundreds of files at the moment, with lots of name-spacing, interfaces, and so forth.

[Derokorian]

What I can tell you, is out of interest I had a look... an old old version of our app had over 21k files, and PHPStorm has always handled it just fine for me. Of course, I have no less than 8GB DDR3 on any work station, and at least dual core i5 or better, with SSDs of course.

$ find . -type f | wc -l
  21003

[dalecosp]

6GB RAM on mine, 1/2 PB PATA RAID, so disk is slower here, at least. Lately it seems I can get it pretty loaded between one Sun VM, Firefox with 14ish tabs, Chrome with a couple dozen, a Word doc or Excel sheet, Outlook, Notepad++ with 30 tabs or so, and Netbeans with hardly anything open at all.

Could be I could throw some stuff away, but who has time for that?

As for the 100K files, I guess I'll stay until I get a better offer, and in the local area 😉 Heck, some days I even stay until after 5.

It was actually 84K; it was the output of "ls -lR | wc -l" in the webroot. So most of those would probably be under /images. IIRC, there's an entire copy of the website underneath it, too, so I guess maybe the real site is more like 40K or 50K files. And then there's that time back in 2009 or something when they made HTML files with some WYSIWYG and stuck them in their own folders to get nice URLS. :rolleyes: No one's seen fit to delete any of those either, and I couldn't be bothered to figure out if NetBeans could exclude certain filetypes or paths (it seems that Beans wants to manage everything, kind of like git without --exclude set anyplace).

I know it handles my original projects pretty well; between NB and git I'm pretty happy with my pet project @work.

I guess it would be interesting to know if someone at eBay or Amazon could load their entire site into one IDE....

[Derokorian]

ls -lR will have WAY more wc -l than the number of files. Check this out:

[[e-mail] test]$ mkdir dir1
[[e-mail] test]$ mkdir dir1/dir2
[[e-mail] test]$ mkdir dir3
[[e-mail] test]$ touch dir1/file1
[[e-mail] test]$ touch dir1/dir2/file2
[[e-mail] test]$ touch dir3/file3
[[e-mail] test]$ ls -lR
.:
total 8
drwxrwxr-x. 3 vagrant vagrant 4096 Jun 10 02:27 dir1
drwxrwxr-x. 2 vagrant vagrant 4096 Jun 10 02:28 dir3

./dir1:
total 4
drwxrwxr-x. 2 vagrant vagrant 4096 Jun 10 02:28 dir2
-rw-rw-r--. 1 vagrant vagrant    0 Jun 10 02:27 file1

./dir1/dir2:
total 0
-rw-rw-r--. 1 vagrant vagrant 0 Jun 10 02:28 file2

./dir3:
total 0
-rw-rw-r--. 1 vagrant vagrant 0 Jun 10 02:28 file3
[[e-mail] test]$ ls -lR | wc -l
17
[[e-mail] test]$ find . -type f | wc -l
3

[NogDog]

Well, I suppose in terms of raw number of files under the web root, that may not be horrible, but hopefully things are a bit compartmentalized, such that you have sub-directories which could be treated as separate projects in your IDE -- or is everything in one huge Git (or whatever) repository? And certainly your IDE would ignore non-source-code files for code analysis, argument-hinting, and so forth, I would hope.

Certainly across our various web servers we have many thousands of files (no idea how many in either raw total or just source code files), but they're compartmentalized into many different Git repositories as individual projects, and at any given time I just have my editor's project set to that specific repo's directory. Of course, I realize you may have inherited something that just grew by amalgamation and now no one has the time or budget to reorganize (been there, done that 😉 ).

[dalecosp]

NogDog;11048317 wrote:

Well, I suppose in terms of raw number of files under the web root, that may not be horrible, but hopefully things are a bit compartmentalized, such that you have sub-directories which could be treated as separate projects in your IDE -- or is everything in one huge Git (or whatever) repository? And certainly your IDE would ignore non-source-code files for code analysis, argument-hinting, and so forth, I would hope.

Certainly across our various web servers we have many thousands of files (no idea how many in either raw total or just source code files), but they're compartmentalized into many different Git repositories as individual projects, and at any given time I just have my editor's project set to that specific repo's directory. Of course, I realize you may have inherited something that just grew by amalgamation and now no one has the time or budget to reorganize (been there, done that 😉 ).

That is certainly the case. Budget ... we've got people running XP on hardware that was current over ten years ago (mostly outside of IT, thankfully). I feel like a one-armed paperhanger more often than not.

I dunno if NetBeans can be configged to ignore certain parts of a import project's file structure. By default it doesn't --- when I thought about putting the site into it, I simply clicked "import new project from remote source", configured the connection and let it go. I gave up after a couple of hours of it grinding away (using most of my system resources in the process). Perhaps I'll look back into it.

It's not likely to go into git anytime though, unless we get rid of the Encoded DB Access class (it was built on top of a commercial package; we've pretty much ended our relationship with that vendor and can't so much as move off of the current box without issues, I think, so no sandbox for it exists. It's pretty harrowing to work on it, sometimes. Let's just say I've gotten pretty good at hackish ways to test new features and functionality on that site ;-) )