working with lots of text files

vaaaska

No code here...just theory and advice...

There is a discussion afoot in my world about building an ultra-simple cms thing for a very specific purpose - displaying work. A database is not really needed...but it could make this easier.

And so, it's about text files. Creating structured chunks that would get called upon as needed. I've never done something to this extent..and so I'm curious about some things.

If I had a file that contained a directory of the files, in essense the database, is there some technique for finding a row and deleting/updating it? I can think of a preg_match option but this seems too crude for this purpose.

I'm not sure about PHP4's ability (on a typical server setup) to handle XML reliably.

Files would likely need to be updated and read/displayed when doing so. From my experience SAFE MODE could kill this. I don't see a way around it aside from going back to the database. Am I wrong about this?

Does anybody know of any 'good' examples of systems that rely upon flat/text files instead of a database?

Merci...

thorpe

Databases are made for storing and querying data. Why make things more difficult than they need to be? If your worried about setting up a database server, why not look into [man]sqlite[/man]? Its simple, small and very very fast.

vaaaska

thorpe wrote:
Databases are made for storing and querying data. Why make things more difficult than they need to be?

Because there are some people who can't do this. The idea is that it would be so simple as uploading the site files, editing a config file of some sort and that's it.

Let's not divert this to a discussion solely about databases...

Jason_Batten

You need to place each piece of information on a new line in seperate files. So columns are new lines and rows are different text files. Or you could seperate them using a symbol of some sort but a new line is SO much easier.

SAFE mode would be a pain no doubt so FTP is the way to go. Check the manual - its is all in there 🙂

This idea is some what, pointless. If you have the option to use a database, USE IT that is why they are there.

Text files are TEXT files. Anyone can read them unless you "do" your security right.

dalecosp

PM Halojoy. He'd love to hear your plans.....

dalecosp

Of course, I've not the slightest idea if he's been around for some time.....

vaaaska

.txt files...

Was thinking about this all night and I think XML would be a better way to go (obviously) but it would require that older servers upgrade their php/xml capabilities (I think - from my experience this would be the case).

XML for the list of files...filed could be .php files (structured chunk as Bubblenut would call it).

The speed issues though with flat files...but the plan is not for large sites with this thing. Quandry...

Still hoping somebody with actual experience doing this in the past turns up with some nice info... 😉

jazz_snob

What exactly will the files hold? Is each file a record, or one file a bunch of records? How big is each record? How often will you need to search them? How often will you need to delete records? Finally, and this isn't a joke, write the app as a web service and have the data stored on a server that does have a database. Now you're talking 😃

[edit]
BTW, I am a fan of text files but usually only if each file is a record. Also, if you never need to delete records, a single text file w/ lots of entries is ok; I have one like that and its is 100KB in size.
[/edit]

vaaaska

Blah, blah, blah...database...the goal is to not use one. End.

Each file would be a record. Each record would be really quite small - 4 or 5 average paragaphs at the most. They would likely be stored at php files and contain an array with our info. Searching? - no searching needed. Delete? - sometimes, but not often.

Questions answered...

Normally I would opt for using a database, but this is a different approach - guided by low user knowledge of php/databases. It should work on any server that will not kill it with safe mode restrictions.

Guess I'll go search Hotscripts for a system that uses flat files...but I thought maybe there might be something that stands out.

Weedpacket

Since this is a CMS, then we are talking a more document-oriented environment than a data-oriented one. XML can work in both, but it was really developed from a document-oriented perspective (I wonder how much documentation is still written in SGML? It's probably like COBOL - huge tracts of the stuff but it's all locked away behind the corporate ice. Anyway...) The two orientations are different enough that there are two document specification formats; DTDs, as the acronym implies, describes document-oriented XML (in which text is just text), while XML Schemas are for data-oriented XML (in which data comes in different types).

That said, there are so many issues involved with developing a document file format that using an existing one (XML) means that a lot of the boring stuff is a solved problem.

With XML it sounds a bit like Apache's Lenya project; but (a) that's written in Java, and (b) requires a vast amount of framework baggage (viz. Cocoon).

jazz_snob

vaaaska wrote:
Each file would be a record. ... Searching? - no searching needed. ... Delete? - sometimes, but not often.

Good candidate for flat files then. I have a CMS that uses flat files but its in perl and kinda wierd...you make all the changes then 'compile' the output and it writes all your HTML files, or just the ones you changed if you tell it. Then you deploy the static HTML pages in your docroot (actually it copies them for you). The user defines 'blocks', each of which is a separate file. Then they can combine blocks to form a page. A typical page has a header block, menu block, footer block, maybe a few ad blocks and the content itself. Uses pattern replacement, so when it parses the page and sees @@@header.html@@@ it knows to grab and insert the header.html file at that point. Its not luser friendly...user has to be competant w/ writing html and css.

vaaaska

PHP4 friendly XML stuff = http://minixml.psychogenic.com (allegedly easier to use than PHP5's SimpleXML functions which is kind of nice to hear - plus it doesn't require any kind of extra installations).

Sites created with this thing I'm talking about would never get too large (more than 200 records I bet) so I don't think grabbing and parsing through a flat file XML database thing would ever present much of a load.

Now I'm kind of excited to tackle this...

bubblenut

If you're going to be using XML have you thought about using XSL/XSLT? Could simplify things quite a bit.

vaaaska

I'm not completely sure about that. The idea is that normal folks will be able to mark things up (with css). Can XSLT and CSS coincide peacefully in this scenario (I'll take a look too)? Anyways, it appears that this has some speed and a discusssion will go down over the weekend. I'm rather excited but I can't really say why...

bubblenut

vaaaska wrote:
Can XSLT and CSS coincide peacefully in this scenario

Definately

madwormer2

Serialize!

Okay, I overuse it, but it's so useful

Weedpacket

bubblenut wrote:
Definately

Agreements; CSS is independent of HTML. Note that Mozilla-based applications use an XML dialect for UI layout, and then style it with CSS (that's why developing themes is so straightforward):

toolbarbutton.bookmark-item:hover,
toolbarbutton.bookmark-item:hover:active,
toolbarbutton.bookmark-item[open="true"] {
  padding: 2px 3px 0 3px;
}

#privacyCategories > expander[open="true"] {
  background-color: #DDE8F3;
  border: 1px solid;
}

"toolbarbutton" and "expander" are of course the names of XUL elements.

And then of course there are the DTDs that store all the strings used in menus and dialogues and whatnot.

<!ENTITY pref.buttons "Selecteer de knoppen die moeten getoond worden in de werkbalk">

[which would be used as, e.g., '<caption label="&pref.buttons;"/>']

vaaaska

Huminhuminahuminahumina...

http://radar.oreilly.com/archives/2006/04/database_war_stories_2_bloglin.html

Rodney_H_

Since this IS a php forum, I thought I would share my two-bits...

When I first started learning PHP almost a year ago, Halojoy's notions of flat files being "just as good if not better" than DBs got me thinking... SO...

I built my first flat-file site.

Nope, I didn't use .txt files.

Nope, I didn't use .xml files.

Nope, I didn't use .dat files, etc, etc...

I used PHP files. AND wrote data to them using keyed arrays.

Each file was of a particular topic.

Each file was very easily edited and maintained.

Only needed files were included into the main document AS needed.

I still think this was one of the most fun projects that I had...

(The site is still up, but hasn't been maintained in a looong time, but am proud of it... 🙂

PS: the content was safe from prying eyes, cos the PHP files could not be viewed if looked at in a browser by themselves, and the files were easily protected by placement on the server, too...

vaaaska

I've got one simple question* regarding the flat-file approach...

If you are on a server that has all the dandy SAFE MODE trimmings won't it be impossible to keep a flat-file or xml database? If I go this route, it must work in most setups...meaning, probably, MySQL.

*I think the last time I said this I realized that it was possibly the stupidest thing I could have said. 😉