warlock->sorcerer->necromancer->wizard

sneakyimp

I have realized this year that I'm pretty tired of validating forms and inserting database records and outputting HTML or JSON or whatever. Seems its time to hit the tomes and treatises and bend the mind to some new magick. What do we think is hot and lucrative? What do we think is horrifying? What do we think is really actually kind of enjoyable? What do we think feels like the quickening and lightning flying from the fingers? Some anecdotes...

Javascript
In a fog of drink I spoke to a coder whom I deem quite my junior and who is a vagrant of sorts. He never seems to have a full-time job but has quite a bit of success bouncing from project to project in some freelance capacity, aided by recruiters. I asked him what was hot these days and he said "Javascript." I scoffed but he insisted. I asked if it was lucrative and with a straight face he told me he was offered $150 an hour for a Javascript position :quiet:. I do see that it appears to be in demand[url] (but weird to see SQL on there?). Bonus points: you can code for servers (see [url=https://nodejs.org/en/]node) and I've heard you can code for mobile devices with something like NativeScript or PhoneGap.

Multithreading/Multiprocessing/Parallel Programming
I've always been drawn to and fascinated by the prospect of unleashing an army of processing power on a computational problem. Infoworld identifies multithreading as one of the most vexing problems in programming. I was fortunate enough to get paid to develop some MT code which, although time-consuming and difficult to write, has paid for itself through its power and efficiency by reducing the number of servers a client needs to run their business. If anyone can point me to a general solution to the problem of resource acquisition in MT coding, I'd greatly appreciate it. I spent time googling solutions to the The Dining Philosphers Problem but was unable to achieve the awakening necessary to share resources between the threads. I have the vaguest recollection from uni that message passing is the best solution, but it's all so fuzzy. I also wonder if this is really a lucrative skill or if the need for it is unusual. Is this esoteric? Or vital?

Nonrelational DB, weirdo DB-like stuff
There are a bunch of these like SOLR, NoDB, NoSQL, BigTable, SimpleDB, et. al. Some claim infinite scalability, but I have some vague inkling of their inability to perform JOINs -- or something like that.

Encryption, old-school C/C++
I've had Bruce Schneier's Applied Encryption for a couple of years now and have been meaning to work through the whole beast, but it's pretty heady, mathematically-speaking. I've thought for years that folks will want privacy from all the grasping, groping corporations and nefarious governments. This seems more urgent than ever now that we've elected Orange Hitler over here in the states. I've also thought that high-performance, low-level code like C/C++ might be quite lucrative, but the work perhaps hard to find. That said, my grasp of manual memory management is not optimal.

And of course I plan to embrace PHP 7, as soon as I upgrade my workstation to Ubuntu 16.

Thoughts?

dalecosp

Heh, just someone to call back on a remote position from SO would be nice.

I'm getting more into JS, and the native stuff is great and JQuery is pretty easily done, but the sheer number of competing paradigms, stacks, and frameworks (and the lack of "proper" documentation?!!??) makes my head spin and make the Laravel/Cake/Zend universe look like a sandbox by comparison.

As far as "weird DBs", I've played with Mongo a little. Don't have a good use case to see what the fuss is about.

NogDog

From what I've seen, the "noSQL" stuff (we use SOLR extensively where I work, but not in the stuff I work on) is good for optimizing read time, for cases where you know exactly what sort of searches you'll be doing (and on large enough data sets where it matters). We have search indexes set up for specific searches/sorts we know we'll be doing, and as we're read-heavy (way, way more reads than inserts/updates), it works well for us. We run periodic checks (daily down to every few minutes, depending on the application and search index) to determine from the SQL DB what needs to be updated in the noSQL index, so it does mean that the data in those indexes may be out of date on occasion, but in our case it's not a big deal, so it works well. If you want to keep it up to date in "real time", it's less efficient than most SQL DBs, so might be problematic in a situation where there is a high amount of inserts/updates.

sneakyimp

So you maintain an old-school SQL database and mirror it in some optimized way using noSQL?

sneakyimp

dalecosp;11058613 wrote:
I'm getting more into JS, and the native stuff is great and JQuery is pretty easily done, but the sheer number of competing paradigms, stacks, and frameworks (and the lack of "proper" documentation?!!??) makes my head spin and make the Laravel/Cake/Zend universe look like a sandbox by comparison.

I'm quite accustomed to and love jQuery and not bad at raw JS. I've written a hierarchical directory component, for instance. I hear "the kids" these days are eschewing jQuery because it's "too heavy." Easier to do now that browsers actually behave themselves.

dalecosp;11058613 wrote:
As far as "weird DBs", I've played with Mongo a little. Don't have a good use case to see what the fuss is about.

I can't quite understand either. Nog mentions "read time" but I'm not sure what he's getting at. I wonder if there might be any mind-opening examples to illustrate the difference or why you'd prefer one over the other?

NogDog

sneakyimp;11058639 wrote:
So you maintain an old-school SQL database and mirror it in some optimized way using noSQL?

Essentially, yes. Or, at least in the specific situation that I'm involved in, where I work on the app that works with the PostgreSQL DB, while the separate search app that makes use of that data pulls the parts its interested in into a SOLR collection -- along with related data from some other data sources -- so that the app using the SOLR collection can perform optimized searches for its particular needs over some rather large data sets.

The "old school" DB provides flexibility (and for most of our uses is fine performance-wise), while the customized search collections in SOLR are optimized for a specific use, which works great for that use, but may not be of any use if you suddenly have some other type of search you'd want to do on it (until you re-design and repopulate that collection, or create a separate collection for that new need).

And take all that with a grain of salt, as what I know about it is based on being close to it, but not actually working with it -- I just provide some API endpoints for them to use to update their collections from "my" data. 🙂