Need XML parsing help - encoded characters breaking

[bunner_bob]

I'm trying to parse an xml file that contains encoded special characters. The parser is chopping off the result and I only get the part AFTER the special character.

For example, if I try to display a node containing:

'Every time you try, you’ll have great results.' (that's &#8217[semicolon] - a curly apostrophe)

What the parser returns is:

'll have great results.'

(Everything up to and through the encoded character doesn't return in the output array).

I'm using a parser class I found and cleaned up a bit. Maybe there's something in there that's causing the problem. Or maybe I need to somehow escape or convert those numeric encoded characters to something else before attempting to parse them. I don't know that much about parsing XML.

Here's the class:

class xmlParser{

// *** ----------------------------------------------------------------
// DECLARATION
var $xml_obj = null;
var $output = array();


// *** ----------------------------------------------------------------
// CONSTRUCTOR
function xmlParser()
{
	$this->xml_obj = xml_parser_create("UTF-8");
	xml_parser_set_option($this->xml_obj, XML_OPTION_TARGET_ENCODING, "UTF-8");
	xml_set_object($this->xml_obj,$this);
	xml_set_character_data_handler($this->xml_obj, 'dataHandler'); 
	xml_set_element_handler($this->xml_obj, "startHandler", "endHandler");
} 


// *** ----------------------------------------------------------------
function parse($xmlFile)
{
	$fp = fopen($xmlFile,"r");
	$xml = fread($fp, filesize($xmlFile));
	fclose($fp);


	if (!xml_parse($this->xml_obj, $xml))
	{
		die(sprintf("XML error: %s at line %d",
		xml_error_string(xml_get_error_code($this->xml_obj)),
		xml_get_current_line_number($this->xml_obj)));
		xml_parser_free($this->xml_obj);
	}


	return true;
}


// *** ----------------------------------------------------------------
function startHandler($parser, $name, $attribs)
{
	$_content = array('name' => $name);
	if(!empty($attribs))
	{
		$_content['attrs'] = $attribs;
	}
	array_push($this->output, $_content);
}

// *** ----------------------------------------------------------------
function dataHandler($parser, $data)
{
	if(!empty($data))
	{
		$_output_idx = count($this->output) - 1;
		$this->output[$_output_idx]['content'] = $data;
	}
}

// *** ----------------------------------------------------------------
function endHandler($parser, $name)
{
	if(count($this->output) > 1)
	{
		$_data = array_pop($this->output);
		$_output_idx = count($this->output) - 1;
		$this->output[$_output_idx]['child'][] = $_data;
	} 
}
}

Any help or suggestions would be greatly appreciated!

• Bob

[bunner_bob]

I solved one problem but found another.

Apparently the parser can work in multiple passes, rather than hitting a chunk of data all at once. So you must concatenate your results. The new dataHandler function:

function dataHandler($parser, $data)
	{
		if(!empty($data))
		{
			$_output_idx = count($this->output) - 1;

		if (!empty($this->output[$_output_idx]['content']))
		{
			$this->output[$_output_idx]['content'] .= $data;
		}
		else
		{
			$this->output[$_output_idx]['content'] = $data;
		}
	}
}

So NOW what's happening is, the characters I encode as right single quote (&#8217[semicolon]) are displaying as something totally different: ’

I'm typing the parser's input and output as UTF-8, and the XML file is UTF-8. So what the heck is going on?

[bunner_bob]

Here are the files - one containing the relevant PHP, the other a test XML file.

According to BBEdit, the XML file is "Unicode (UTF-8, no BOM)". I also tried plain UTF-8. Didn't make a difference.

In either case, the numerical entity for a curly apostrophe is coming out as ’. Other numerical entities are similarly messed up.

I would be immensely grateful if anyone could shed some light on this.

[bunner_bob]

OK - worked out a brute force solution.

Apparently the parser doesn't mind verbose entities (or whatever they're called) - i.e. '’' for a single quote - IF you double encode the ampersand.

Since my XML has to work in Flash as well as PHP (for the HTML version of the site) I'm writing it all using numeric entities. Then I made a pair of arrays - one numeric entities and one verbose - and use str_replace() to replace them all when using PHP. And then a second str_replace to encode the ampersands.

So '&8217[semicolon]' ends up as '’'

And it works.

Clunky as hell, but it works.

$numerics = array( // of course this board is encoding my numeric entities...
					'Á',
					'á',
					'É',
					'é',
					'Í',
					'í',
					'Ó',
					'ó',
					'Ú',
					'ú',

				'’'
			);

$verbose = array (
					'Á',
					'á',
					'É',
					'é',
					'Í',
					'í',
					'Ó',
					'ó',
					'Ú',
					'ú',

				'’'
			);


$xml = str_replace('&', '&', str_replace($numerics, $verbose, $xml));