I have a space delimited file that I need to import into a database. I've done this before with other space delimited files, but before each row of data was contained on a single line. At the command line, I'd use the following:

mysql -u <user name> -p<password> -e "LOAD DATA INFILE '/<location of file>/<filename>' INTO TABLE <table name> FIELDS TERMINATED BY '' IGNORE 12 LINES" <database name>

I've set up my database structure where each column in the database matches the proper amount of spaces for each field. When I run the above command, it loads the data flawlessly.

In this file, there are multiple lines per entery and I need to know what to do to import those into my database. What would be the recommended way of doing this?

Here's an example of one of the enteries:

S1234567 123456 WATER IN GEAR OIL                        01/10/06 03808 240000159 01/20/06 WARRANTY      93-7977
                                                              O-RING                        1 EA   1.1310   1.13
                                                              REGULATOR-PRESSURE, MANUAL    1 EA  98.5660  98.57
                                                              O-RING ATL                    1 EA   3.1850   3.19
                                                              SCREW-PPH                     1 EA   0.3250   0.33
                                                              NUT-WELL                      6 EA   0.9620   5.77

These would need to be imported into a database as such:

[table1]
+-------------+-------+--------------------+-----------+------+--------------+-------------+---------+--------+-----------------------------+----+-----+----------+--------+
|order_number |custno |trouble_description |order_date |eq_no |serial_number |promise_date |job_type |prod_no |part                         |qty |unit |item_cost |cost    |
+-------------+-------+--------------------+-----------+------+--------------+-------------+---------+--------+-----------------------------+----+-----+----------+--------+
|S1234567     |123456 |WATER IN GEAR OIL   |01/10/06   |03808 |240000159     |01/20/06     |WARRANTY |93-7977 | O-RING                      |1   |EA   |   1.1310 |   1.13 |
|             |       |                    |           |      |              |             |         |        | REGULATOR-PRESSURE, MANUAL  |1   |EA   |  98.5660 |  98.57 |
|             |       |                    |           |      |              |             |         |        | O-RING ATL                  |1   |EA   |   3.1850 |   3.19 |
|             |       |                    |           |      |              |             |         |        | SCREW-PPH                   |1   |EA   |   0.3250 |   0.33 |
|             |       |                    |           |      |              |             |         |        | NUT-WELL                    |6   |EA   |   0.9620 |   5.77 |
+-------------+-------+--------------------+-----------+------+--------------+-------------+---------+--------+-----------------------------+----+-----+----------+--------+

Thanks in advance for any assistance offered.

    This is just a guess, but...

    FIELDS TERMINATED BY ''

    There's no space between the ' and '.... If they're space delimited, you'll need a space in there.

      That may be so, but it still imported the file that I needed into the database with no problems. I'll try it both ways to see if it continues to work.

      However, I'm looking for guidance on how to import the new file, which I've given an example of the text. Each of the six lines would belong to the same order_number.

      I'm just not sure what way to go here. The books that I have don't talk about multiple lines in a text file, like what I need, and other sites I've looked at doesn't have any information on this. All their examples are for text files where each entry's data is on the same line. I'm unable to find anything at the mysql web site and the forums there. I've been searching for the solution to this the better part of the day now.

        kconley:

        There is one thing you might try and that is to do a search either in the MYSQL Manual or Google for the term "chunk". This allow you to add (as the name implies) chunks of data into a database field. This might work, but it is going to be a constant nightmare.

        You stated:

        I've done this before with other space delimited files, but before each row of data was contained on a single line.

        And that is why it worked. Just as a space delimited file only works with single lines, a database file only works with single rows.

        You could add additional fields to your database for part1, part2, ..., part 150, but this would creat a rather large database with lots of redundant information. Also, one work order might use 2 parts and the next one would use 63 parts. You would be re-entering the same part numbers over and over and thus defeating the main purpose of a database.

        It appears to me that what you are trying to do here is to create some sort of "work order" program or "invoice" program. If that is what you are trying to accomplish then the key to your success is going to be in your database design and the relationships you establish between the tables.

        You are going to need to create several tables such as customers, work_orders, work_order_details, parts, suppliers, invoices, etc., etc.. Then you need to set up Primary Key/Foreign Key realtionships between the tables and normalize your tables.

        After you have completed your database design, established the relationships, and normalized your tables you can move on to create whatever forms you need such as Invoices, Work Orders, etc..

        Then you write the PHP code to extract information from your database tables and display it on your form or update your database or whatever else you may want to do.

        This probably sounds like a lot of work and it is, but I don't know of any other way to do it.

        Maybe there is some PHP expert out there who can tell you how to rub two keyboards together and make a program instantly appear on your screen, but I don't think so.

        Anyway, Good Luck

          Thank you, AIS4U for the good information. I'll check into “chunk”. I've also thought that perhaps there may be a way for PHP to parse the text and put it into the database that way. I just don't know where to start with parsing a text file via PHP since I'm still new to both PHP and MySQL.

          I do have my work cut out for me on this project. Here's the scenario:

          My company uses a proprietary database system which is basically the heart and soul of the company. It keeps track of every nut and bolt in inventory, plus every penny earned and spent. It almost does everything you could need. Almost.

          We've tried to ask the vendor how to get my web server to communicate directly to the database, and we were basically laughed at. The company has used this system for so long and literally has millions of dollars invested in it, it's not as simple as moving everything to a more open platform.

          One of the things the system does not do is allow our service department to schedule their jobs more efficiently.

          Our service department deals with warenty repairs, new item set ups and customer's non-warenty repairs. With several technicians, we have one that's dedicated to setting up only new equipment, where the others do repairs, or set up new equipment when there are no repair jobs to do.

          I've been tasked with creating a "Scheduling Tool" for the service department.

          On our proprietary database, we are able to run reports which output most of the data we need, and we've used this method to create other resources which our service department is currently using. With the previous jobs, we were able to alter our reports to output everything on a single line for each job. The data is saved to a text file where it's FTP'd to my web server and imported into the MySQL database. The reports are all scheduled on the main server, and I've writen a script that does the FTP and data import which runs on a cron job. These reports are all space delimited. This time the data we are trying to get will not export out to a single line per job.

          We've asked the vendor how to alter this and our reply was, "We don't allow customers to alter these reports."

          In this Scheduling tool, I need to display as much job data as possible to minimize user input, while allowing the service manager to schedule tasks to the techs.

            kconley:

            Well, it sounds like you certainly have a real challenge ahead of you.

            After reading your last posting maybe the "chunk" thing isn't the best way to go.

            I'm certainly not a PHP expert by any means. Most of my application development has been with Visual Basic 6 and Access or SQL.

            However, there is one PHP string function that might be of some value to you as far as getting the information you are able to retrieve into some workable format.

            I don't know if you have used strtok( ) or not, but I used it with the "example of one of the enteries" you listed in your first posting and was able to break the string up into almost a useful format. I think with a little more fooling around with it and it just might work.

            Here is a little blip of info about it:

            Tokenizing and Parsing Functions
            Sometimes you need to take strings apart at the seams, and you have
            your own notions of what should count as a seam. The process of
            breaking up a long string into words is called tokenizing, and among
            other things it is part of the internals of interpreting or compiling any
            computer program, including PHP. PHP offers a special function for
            this purpose, called strtok().
            The strtok() function takes two arguments: the string to be broken
            up into tokens and a string containing all the delimiters (characters
            that count as boundaries between tokens). On the first call, both
            arguments are used, and the string value returned is the first token.
            To retrieve subsequent tokens, make the same call, but omit the
            source string argument. It will be remembered as the current string, and the function will
            remember where it left off. For example
            $token = strtok(
            “open-source HTML-embedded server-side Web scripting”,
            “ “);
            while($token){
            print($token . “<BR>”);
            $token = strtok(“ “);
            }
            gives the browser output:
            open-source
            HTML-embedded
            server-side
            Web
            scripting
            The original string would be broken at each space. At our discretion, we could change the
            delimiter set, like so:
            $token = strtok(
            “open-source HTML-embedded server-side Web scripting”,
            “-”);
            while($token){
            print($token . “<BR>”);
            $token = strtok(“-”);
            }
            This gives us (less sensibly):
            Open
            source HTML
            embedded server
            side Web scripting
            Finally, we can break the string at all these places at once by giving it a delimiter string like
            “ -”, containing both a space and a dash. The code:
            $token = strtok(
            “open-source HTML-embedded server-side Web scripting”,
            “ -”);
            while($token){
            print($token . “<BR>”);
            $token = strtok(“ -”);
            }
            prints this output:
            open
            source
            HTML
            embedded
            server
            side
            Web
            scripting

            Notice that in every case the delimiter characters do not show up anywhere in the retrieved
            tokens.
            The strtok() function doles out its tokens one by one. You can also use the explode() function
            to do something similar, except it stores the tokens all at once into an array. After the
            tokens are in the array, you can do anything you like with them, including sort them.
            The explode() function takes two arguments: a separator string and the string to be separated.
            It returns an array where each element is a substring between instances of the separator in
            the string to be separated. For example:
            $explode_result = explode(“AND”, “one AND a two AND a three”);
            results in the array $explode_result having three elements, each of which is a string: “one
            “, “ a two “, and “ a three”. In this particular example, there would be no capital letters anywhere
            in the strings contained in the array, because the AND separator does not show up in
            the result.
            The separator string in explode() is significantly different from the delimiter string used in
            strtok(). The separator is a full-fledged string, and all its characters must be found in the
            right order for an instance of the separator to be detected. The delimiter string of strtok()
            specifies a set of single characters, any one of which will count as a delimiter. This makes
            explode() both more precise and more brittle—if you leave out a space or a newline character
            from a long string, the entire function will be broken.
            Because the entire separator string disappears into the ether when explode() is used, this
            function can be the basis for many useful effects. The examples given in most PHP documentation
            use short strings for convenience, but remember that a string can be almost any
            length—and explode() is especially useful with longer strings that might be tedious to
            parse some other way. For instance, you can use it to count how many times a particular
            string appears within a text file by turning the file into a string and using explode() on it,
            as in this example (which uses some functions we haven’t explained yet, but we hope make
            sense in context).
            <?php
            //First, turn a text file into a string called $filestring.
            $filename = “complex_layout.html”;
            $fd = fopen($filename, “r”);
            $filestring = fread($fd, filesize($filename));
            fclose ($fd);
            //Explode on the beginning of the <TABLE> HTML tag
            $tables = explode(“<TABLE”, $filestring); // assumes uppercase
            //Count the number of pieces
            $num_tables = count($tables);
            //Subtract one to get the number of <TABLE> tags, and echo
            echo ($num_tables - 1);
            ?>

            The explode( ) function might also be helpful to you:

            The explode() function has an inverse function, implode(), which takes two arguments: a
            “glue” string (analogous to the separator string in explode()) and an array of strings like
            that returned by explode(). It returns a string created by inserting the glue string between
            each string element in the array.
            You can use the two functions together to replace every instance of a particular string within
            a text file. Remember that the separator string will vanish into the ether when you perform an
            explode()—if you want it to appear in the final file, you have to replace it by hand. In this
            example, we’re changing the font tags on a Web page.
            <?php
            //Turn text file into string
            $filename = “someoldpage.html”;
            $fd = fopen($filename, “r”);
            $filestring = fread($fd, filesize($filename));
            fclose ($fd);
            $parts = explode(“arial, sans-serif”, $filestring);
            $whole = implode(“arial, verdana, sans-serif”, $parts);
            //Overwrite the original file
            $fd = fopen($filename, “w”);
            fwrite($fd, $whole);
            fclose ($fd);
            ?>

            I have also had the thought that you might be better off using some kind of a flat file instead of a database. A flat file should work ok unless you have 100's of these schedulings each day. And, depending on what else you might want to do with the data.

            I will let this rattle around in my little pea-brain over the week-end and see if I come up with anything more I can suggest to you.

            My background isn't computer programming it is in the legal profession. I spent over 15 years practicing law. I'm not an expert in intellectual property law, but my opinion would be that while your software provider does not have to give you access to their source code or the program they have created to manipuate the data in the database, the information in the database belongs to you and they would have to make the database accessible to you. However, that is a subject best left for another day and perhaps a different forum.

            Good Luck

              Write a Reply...