parsing PHP itself

sfullman

the only thing I really need to do is arrays, but the critical issue is knowing where encapsulated strings begin and end. Consider a case where advanced users can enter PHP code settings in array format in a textarea, as follows:

Define settings for this dataview:

'subtables'=>array(
   /* this is user comments with array('value1'=>5, array(0,56,39,100)) in it - we want to ignore it and any 'subtable'=>$var type stuff until AFTER we get out of this comment HERE-> */
   'value1'=>5
   'type'=>'table',
   'value2'=>'etc.',
),
'columns'=>array(
   'firstname'=>array('value'=>'{default}', 'field'=>'input'),
   'lastname'=>array('value'=>'{previous_value}', 'field'=>'select'),
),

I like the idea that the user can add comments and indent as feels comfortable to them. BUT here is the problem; I want to be able to modify this user-typed array, keep their comments, but replace nodes (like modify columns node, add a field array to it, or modify even deeper in the array), while keeping the user comments and indentation.

I'm thinking a parser would need to declare some secondary reference array of starting and ending ranges for a given node or subnode, and not only that, would have to not be fooled by comments that might "LOOK" like ending points.

Anyone have any suggestions? Again the goal is 1) make fine-grained and surgically precise edits to array-like PHP coding and 2) preserves any comments and existing indentation as much as possible. Thanks for your help!

Sam

bradgrafelman

Why does it have to be PHP coding? For example, this:

[subtables]
value1=5
type="table"
value2="etc."

[columns]
firstname[value]="{default}"
firstname[field]="input"

lastname[value]="{previous_value}"
lastname[field]="select"

seems quite readable (and even more non-programmer friendly than editing a PHP script), and, when fed into [man]parse_ini_file/man, produces the array structures you'd expect:

Array
(
    [subtables] => Array
        (
            [value1] => 5
            [type] => table
            [value2] => etc.
        )

[columns] => Array
    (
        [firstname] => Array
            (
                [value] => {default}
                [field] => input
            )

        [lastname] => Array
            (
                [value] => {previous_value}
                [field] => select
            )

    )

)

sfullman

well the quick reason is that anyone using my script is already familiar with php, AND I hate having to learn one more thing I have to store in my brain. (Just to get the point across, I'm going to write the rest of this post in Lithuanian 🙂

the other issue is, I want to retain a) the comments both //this type and #this type and / this type / inline. What I really need is something that returned an array like this:

array(
   array(
      'node'=>'asdf',
      'start'=>135,
      'stop'=>178,
      'valuestart'=>142,
      'valuestop'=>170,
   ),
   array(
      etc.
   ),
),

where node=the name of the array node, and 135-178 represents the entire "chunk" of coding, and also valuestart and valuestop may (alternately) represent the expanse of the string value. With these numbers I can snip out, modify, and re-insert code string and theoretically keep the comments and existing indentation.

consider phpMyAdmin, haven't looked but they must have a mighty fine SQL parser to be able to submit a gazillion sql queries in a textarea and process it; you know they've got to split each one up with a semicolon.

the goal here is to access user- written code via a textarea OR via a form defining options, and have the form defining options minimally effect what the user wrote, if present.

What I thought was, maybe someone has a function that would produce that mapping array like I showed above.

laserlight

sfullman wrote:
well the quick reason is that anyone using my script is already familiar with php, AND I hate having to learn one more thing I have to store in my brain. (Just to get the point across, I'm going to write the rest of this post in Lithuanian

Why don't you use Lithuanian for the configuration script then?

Yes, I'm kidding, but not entirely: how much of PHP are you going to parse? Chances are, you're going to settle for a small subset of PHP, so simple that someone without knowledge of PHP could learn it. In that case, the "person already knows it" argument falls flat: anyone could learn "ini" file format, or JSON (or perhaps YAML, since JSON does not have comments).

sfullman wrote:
consider phpMyAdmin, haven't looked but they must have a mighty fine SQL parser to be able to submit a gazillion sql queries in a textarea and process it; you know they've got to split each one up with a semicolon.

They could always perform a multi query, thus shifting the work of parsing the SQL statements to MySQL.

sfullman

good one on the Lithuanian laserlight 🙂

first off, the "user" in this case is "me', but also any developer who works this project.

The issue is that some of the arrays are quite involved and being in development I/we would want to store comments. Also, the forms for managing the attributes are jquery-intensive so while THEY are being developed I still have some time to hand-manage the coding.

and I would probably like to store something like this as well:

<?php
array(
   'node1'=>5,
   'node2'=>function(){
      //blah blah blah
   }
),
?>

also this goes with my philosophy of "make the data management sophisticated so the user can be simple - because they will be"

sfullman

laserlight;11021143 wrote:
They could always perform a multi query, thus shifting the work of parsing the SQL statements to MySQL.

oh, on this, one of the greatest features of phpMyAdmin is the ability to paste a whole mess of SQL queries into a textarea, hit submit, and have it work. again, goes along with my philosophy of user interface 🙂

laserlight

sfullman wrote:
The issue is that some of the arrays are quite involved and being in development I/we would want to store comments.

The format recognised by parse_ini_file() allows for comments, so that is not a problem. As I mentioned earlier, while JSON does not allow for comments, its close relation YAML does, and YAML parsers are available for PHP.

What might be a problem would be trying to recreate the configuration text with its original formatting, but modified in content. However, do you really need to do that since you can store the modified configuration elsewhere?

sfullman wrote:
and I would probably like to store something like this as well:

Now this is problematic: you actually want to evaluate the configuration file as PHP, and not merely as a "dumb" configuration? This potentially introduces security vulnerabilities, but it is quite possible: you have your script write the configuration into a PHP file (with assignment to a variable) that is then included elsewhere. Then that PHP script that includes this generated PHP include file just makes use of the designated variables.

In such a case, I posit that you just give up on the idea of this being "a case where advanced users can enter PHP code settings in array format in a textarea": just concede that this is a case where a programmer who knows what he/she is doing is setting PHP code settings in a configuration file that is a PHP include file. Trying to "make the data management sophisticated so the user can be simple - because they will be" is dumb when the user will obviously not be simple since you're setting it up such that users must be skilled. If you do really want to follow your claimed philosophy, then you should stick to a simple format like ini, or a somewhat more technical but still simple format like YAML, without giving the user the power to actually write executable code.

sfullman wrote:
one of the greatest features of phpMyAdmin is the ability to paste a whole mess of SQL queries into a textarea, hit submit, and have it work. again, goes along with my philosophy of user interface

Sure, but my point is that phpMyAdmin does not need to "have a mighty fine SQL parser" itself to do this, at least not with the evidence that you're presenting.