• PHP Help PHP Databases
  • [RESOLVED] Mysqli Transaction Example and a stable way to import data from a file into Mysql

Hello All:

Firstly let me express my relief at finding a PHP board where the community is active. On my last thread I got some great help here. Encouraged by that I am posting my next programming dilemma.

I have a fair amount of data in offline data entry modules in Ms Access. I need to get this data into mysql in a transaction safe way.

I was thinking of using mysqli transactions. Also what is a good format (csv, xml...) to use to export data so my php script can open the file and in a transaction safe way import into mysql.

I dont think loadinfile will do it, because in case there are errors the whole batch will have a problem. I am thinking of doing it one record at a time...

Any code or ideas regarding mysqli transactions or importing data into mysql via a script in a transaction safe way will be appreciated...Thanks...

    Ok Ive been at this for a while now.

    I am trying to load a csv file intol Mysql using PHP. Here is my concern.

    I have been able to do it by running this sql:

    $sql = "LOAD DATA LOCAL INFILE '" . $uploadDir . $csvfile . "' INTO TABLE " . $table ." FIELDS TERMINATED BY ',' ENCLOSED BY '\"' LINES TERMINATED BY '\r' IGNORE 1 LINES";

    However i am concerned about this being non transaction safe. However when I wrap this in a transaction, I am not able to do a ROLLBACK. It just doesnt roll it back.

    Any ideas on how to wrap this in a transaction. Or should i be reading each row individually and inserting it individually and wrapping that in a transaction?

      arjuna;10946597 wrote:

      However i am concerned about this being non transaction safe. However when I wrap this in a transaction, I am not able to do a ROLLBACK. It just doesnt roll it back.

      Any ideas on how to wrap this in a transaction. Or should i be reading each row individually and inserting it individually and wrapping that in a transaction?

      Arjuna.
      We had a topic on Transactions a short while ago.
      I did, like you, some test script to use PDO (and/or MySQLi) classes
      to do the rollback.
      http://usphp.com/manual/en/function.PDO-rollBack.php

      I realy recommend using PDO before MySQLi.
      I wont go into details here, but I say PDO is the nice tool I need for working with database.

      After much testing and never getting rollback to work at all,
      I happend to find the reason in the web.
      MySQL uses by default and many times ONLY MyISAM tables.
      The drawback with this is:

      - MyISAM does not support TRANSACTIONS
      - InnoDB engine supports it, and MySQL has got this choice to.

      However, to enable InnoDB tables, you can need to change in your my.ini
      my.ini is the important settings file for MySQL,
      like php.ini is for PHP
      (In Linux my.ini is called my.conf, I think)

      The lines that can effect how tables are created are, in my.ini

      # The default character set that will be used when a new schema or table is
      # created and no character set is defined
      default-character-set=utf8
      
      # The default storage engine that will be used when create new tables when
      default-storage-engine=MYISAM
      
      #*** INNODB Specific options ***
      
      
      # Use this option if you have a MySQL server with InnoDB support enabled
      # but you do not plan to use it. This will save memory and disk space
      # and speed up some things.
      
      #skip-innodb

      default-storage-engine=MYISAM
      If nothing else is told in the 'CREATE TABLE' command,
      then this option will determine what Engine is used. MYISAM/INNODB

      #skip-innodb
      This is often used by your MySQL installation, looks like this:
      skip-innodb
      This will disable InnoDB, and ofcourse your possibility to use TRANSACTIONS

      Note:
      After changing you my.ini settings
      you need to stop and restart MYSQL.
      Simplest way is to RESTART, REBOOT Windows

      Here is one example
      when I created my First InnoDB table 🙂

      <?php
      
      // MySQL DATABASE Config
      $dbhost = 'localhost';
      $dbuser = '';
      $dbpass = '';
      $dbname = 'test';
      $dsn = "mysql:dbhost=$dbhost;dbname=$dbname";
      
      try{$db = new PDO($dsn,$dbuser,$dbpass);}
      catch(PDOException $e){exit('PDO: '.$e->getMessage());}
      
      $db->setAttribute(PDO::ATTR_ERRMODE,PDO::ERRMODE_WARNING);
      //$db->setAttribute(PDO::ATTR_AUTOCOMMIT, 0);
      
      
      	$sql = "CREATE TABLE IF NOT EXISTS users (
      	id     INTEGER NOT NULL PRIMARY KEY AUTO_INCREMENT,
      	name   VARCHAR(32) UNIQUE,
      	pass   VARCHAR(40) NOT NULL,
      	level  INTEGER     DEFAULT 1,
      	tjoin  INTEGER     NOT NULL,
      	code   VARCHAR(40) DEFAULT '',
      	email  VARCHAR(50) DEFAULT '',
      	fname  VARCHAR(32) DEFAULT '',
      	lname  VARCHAR(32) DEFAULT '',
      	status VARCHAR(50) DEFAULT '',
      	lang   VARCHAR(32) DEFAULT '',
      	ip     VARCHAR(32) DEFAULT '',	
      	url    VARCHAR(64) DEFAULT '',		
      	tzoff  INTEGER     DEFAULT 0  
      	) ENGINE InnoDB COLLATE utf8_general_ci";
      
      if($db->exec($sql)!==false)
      	echo 'success, table users exists';
      else
      	echo 'failure: '.print_r($db->errorInfo(), true);

      You see I at the end, after fields definitions
      I have 2 TABLE Definitions, engine + collate:
      ) ENGINE InnoDB COLLATE utf8_general_ci";

      Good Luck!
      🙂

        Hi Halojoy:🙂

        Thanks for the detailed response. Essentially I realized the same thing. Transactions dont work with ISAM tables. But they work with InnoDb tables.

        I was simply able to convert my table to INNODb using an ALTER statement in PHPMYADMIN and my lo and behold the transactional features of Mysqli started working.

        But then I ran into the next issue in trying to import my data from a CSV file into my database using LOAD DATA INFILE. It works, for anyone in the future who wants to use it here is the code:

        function doUploadLoadInFile($csvfile,$uploadDir,$table,$uniqueId,$dbName) 
              {  
        /* Connect to a MySQL server */ $dbManual = new mysqli('localhost', 'xxxx', 'xxxx', $dbName); $sql = "LOAD DATA LOCAL INFILE '" . $uploadDir . $csvfile . "' INTO TABLE " . $table ." FIELDS TERMINATED BY ',' ENCLOSED BY '\"' LINES TERMINATED BY '\r'"; if (mysqli_connect_errno()) { printf("Can't connect to MySQL Server. Errorcode: %s\n", mysqli_connect_error()); exit; } #Count rows in table before upload $sqlCount1="SELECT " . $uniqueId . " FROM " . $table; $result1 = $dbManual->query($sqlCount1); $resultCount1=mysqli_num_rows($result1); // turn off auto-commit mysqli_autocommit($dbManual, FALSE); $result = $dbManual->query($sql); if ($result !== TRUE) { echo ("There was an error in the upload, it is being rolled back!"); mysqli_rollback($dbManual); // if error, roll back transaction } else { echo ("The upload was successful, it has been committed!"); // assuming no errors, commit transaction mysqli_commit($dbManual); } #Count rows in table after upload and calculate the no of rows uploaded. $sqlCount2="SELECT " . $uniqueId . " FROM " . $table; $result2 = $dbManual->query($sqlCount2); $resultCount2=mysqli_num_rows($result2); $NoAffectedRows=$resultCount2-$resultCount1; printf("Affected rows (Upload): %d\n", $NoAffectedRows); // close connection mysqli_close($dbManual); }

        However i realized that LOAD DATA INFILE is not the ideal solution as in case there is an error in uploading a record it will skip it. Therefore i am now redoing this by using individual insert statements wrapped in a mysqli transaction as described in the above code.

        However issues just keep coming up. The next issue is one of Mysqli prepared statements which i will post in a separate thread. These haves several issues in that in that each of the parameters have to be specified to be bound, moreover their data type has to be listed as:

        /* Create the prepared statement */
        if ($stmt = $mysqli->prepare("INSERT INTO CodeCall (FirstName, LastName) values (?, ?)")) {
        
        /* Bind our params */
        $stmt->bind_param('ss', $firstName, $lastName);
        
        /* Set our params */
        $firstName = "Jordan";
        $lastName  = "DeLozier";
        
        /* Execute the prepared Statement */
        $stmt->execute();
        
        

        However I have to run this code dynamically on different tables, with different parameters, each imported from a csv file, and each field has a different data type which i cannot gather from looking at the csv file (as some values are null)

        Therefore mysqli prepared statements seem really restrictive and bothersome and I am thinking of just going with old mysql stlyle inserts where i can simply put in a string a fields and corresponding values and wrap in a loop and in my transaction and it will work like magic.

        However this will be much slower than load in file, but much more reliable. It will be slower than mysqli queries as each query will have to be executed separately unlike in a prepared statement. And i will have to figure out how to guard against mysql injection attacks .

        But it will work. Mysqli prepared statements dont seem suited at all to dyamic parameters with unknown data types.

        Any suggestions?

        Should i be starting a separate thread for this?😕

          Yes,
          start a new thread, if the discussion has turned away from original subject.

          There are people that go on with endless discussion on this and that
          in a topic where initial question is solved in the first few posts.
          This is really not a topic for get help.
          This is more a project asking for partners to develop solution to a task.
          Or wanting to have private teachers' lections daily

          This is no good.

          There are other that ask a straight and specific question,
          get some good answers and mark it resolved.
          And when next question come they start a new topic.
          This is good.

          Regards /halojoy

            Write a Reply...