I think this book can help non-technologists understand the accumulating perils of lock-in, the creeping entropy of our connected world.

Wow, that sounds dark!  But like Lanier, I am enamored, enthralled by, or in thrall to–pick your poison–technology and the smart dissemination of information.  Lanier is funny and entertaining too.  When he is interviewed on TV, the co-panel smiles nervously as he talks.  They have not read his book.

Speaking of lock-in, I really need to go back to thinking about Eclipse and the Google version of Oracle’s java…

Instead, I am talking about the corporate user, for whom documents still matter:  they are legal and they contain private information and they need to be persisted for all time.  These three conditions are true in varying degrees of course, but can be assumed to be true for the sake of argument.  The “document” already seems like an anachronism that the office community would like to think has gone away.  Weren’t we supposed to be moving to the paperless environment any day soon?  But they can’t make it go away.  Documents are still needed.  They are portable pieces of information and portability is still demanded either by the workflow (the law firm in Singapore needs to sign off on the contract but we can’t allow them into Salesforce directly) or by operational realities (the one-pass vendor needs the template to spool into their output stream).  Therein lies the problem:  portability.  Once documents escape and start floating along email chains or get passed from one propritary data island to another through an encrypted or, worse, unprotected pipe, they become clones-at-large, both insecure and very likely multi-versioned.

What about codifying this kind of information in a database and controlling access via your extranet?  This is nice if you can get it, and does work for information sets that are repetative and conform to generalized schemas.  Hmmm, speaking of schema, what about XML?  Same issue:  schema. There are just some processes that are so irregular that database factoring can never accommodate all the exceptions and anomalies.  This is why it has been such a challenge to come up with business XML schemas that everyone accepts as standard.  Interfaces are myriad.  Turns out there are devils in the details, or just a lot of details, things so specific they are not generizable across all units, let alone across all companies.  We can argue about this all day, but any of us who consult in corporate work groups know documents are with us and will be with us for the forseeable future.  One example:  I have built work flow tools for corporate actions teams, the folks who facilitate mergers and aquisitions, and, guess what, every deal is different.  A lot of things are the same, most things even, but not all things.  The common information goes into databases in adressable fields.  The rest goes into documents.  And reorgs involve contracts and letters-of-intent, etc., and at least two if not three legal teams need to review these documents.

A brief ancient history lesson.  Before the fax, there were couriers, legions of them.  Really only one person could have the document at any given time and if people had copies, somehow they knew this and corrections were collated carefully by the paralegal or the admin later at some central point, and this happened in slow motion.  After the fax came along, the process was pretty much the same, just less slow and resulted in eye strain.  Then came email.  Ah, not slow anymore and not hard to read again, but the sort of chaos all office staff recognizes ensued.  Documents went viral and got emailed all over the place.  People tried to keep track of the rules on who should make changes and in what order, but such a process, an unenforceable process based on verbal rules, soon breaks down.  It is like the telephone game where the story passed by whisper from one person to the next ends up bearing no similarity to its first ancestor.  Most of the confusion is not intentional, but confusion is confusion.  The problem is not so much after the event, when the final document gets stored in some agreed-to location, but during the heat of the moment, during the workflow period for the deal or production job in question, when errors and inefficiencies are most expensive and hardest to intercept.

The response to this problem was to approve more informal rules which usually boil down to the same model.  Most often, one person or two persons (for “redundancy”) are put in charge of a private storage location.  And that famous instrument of false security, the Excel spreadsheet, is used as a surrogate database.  In this way, another document, subject to the same limitations as the documents it purports to track is utilized.  One of the hens is minding the hen house.  And worse than that, the human minder of this store is critical.  The Excel “database” (substitute the work group listing or storage tool of your choice) always fails to convey all the information needed to keep things running smoothly.  The rest of what is needed to be known is in the head of the gal in charge, in her head!

Yet, I have not mentioned the thing that is suddenly causing the anxiety.  We have spoken of pain, yes, but not acute panic.  The new factor is security.  The information risk officers all over the corporate landscape woke up one morning recently and realized the document is at large, it has escaped, and it is not secure.  Well, maybe it is.  Maybe it has a password.  If it does, it was added on the honor system.  So it happens most of the time, not always.  And the risk officers also know no one can tell them, really, where the documents have been sent or even where the canonical version is or what its ancestors look like.  Have I mentioned “auditors”?  Let’s not even go there.

The answer of course is a document repository which at a minimum will embody the following features:

• No-brainer access to the latest version of any document
• Search features using categories and document types
• Document profiles for defining each implemented use of base document definitions (vendor-specific versions of templates, for example)
• Version histories and version statuses
• Tracking of distributions and version changes (to whom and from whom)
• Access control based on user roles
• Secure access
• Safe long-term storage in the cloud or the data center

With this feature set, email can be eliminated as the transporter, step one to regaining control.  Email can still used as an alerting mechanism, but rather than containing the document attached, the email links the user back to the secure portal, thus ensuring that the download is recorded and achieved securely.  Confusion can still occur when, for example, two individuals, both reviewing and changing the same document are not aware of one another.  But the variance is automatically detected and available for parsing when the two “published” versions hit the repository.  At this point we are back to the paralegal or line-of-business admin we mentioned in the dark time of paper copies.  He had two copies then.  He could see them.  He now has two published versions.  He can see them too.  The same collation task resolves the variance in either scenario and he commits the fixed version, immediately identified as final.  Final is always known and so are the molecules that came before.

This is all well and good, but we need to take our solution one step further and make it a solution.  Let’s not let the tail wag the dog and forget about users because we are in a panic about security.

We would like to put up a Sharepoint site and call it a day.  But this will not serve users in a complex work flow environment.  Instead, we need to think of the Sharepoint or whatever document storage framework we adopt or build not as an application for users but as an API, a set of interfaces for the workflow applications that already know how to manage the process.  They are good with the process but are not good at dealing with document meta-data.  Whenever a document comes along in one of these apps, we as solution providers need to pass it off to the doc repo to leverage its framework for a standardized and secure document handling process. 

Auditors and worker bees alike will thank you.

More on this framework and its integration into disparate workflows in future posts.

So there I am reading the user manual (such as it is at this point) and trying to get the thing put together.  And it works, but I want to do more advanced things like autocomplete and post actions on mutil-row datasets and the like, so I keep going back to the trough, looking for insights from bloggers and from the usual how-to things we find out there when a technology is still new.  Most of it is the simple stuff that doesn’t make it in the real world.  Discussions that fail to make the arching connections between one thing and another.  And fail to answer 85% of one’s advanced questions.  Microsoft likes to set up a myriad of pathways toward the same result and then sit back and watch the people in the field sort things out.  There is an implied competition among the pathways, though the evangelists usually lay out the official best practice.

Oh, yeah, my random thought and it comes by way of just such an evangelist.  (Trust me I am not being derogatory–these people are generally smarter than I am and know what they are talking about and seem all-knowing in an only slightly eerie way.)

So my thought–which for me I may as well admit was practically an epiphany–comes from watching a how-to video on MVC 2 by Scott Hanselman.  The video is good and I learned more than a few things but what struck me most of all was a comment he made toward the end of the presentation.  He said (I am paraphrasing) that while web forms speaks to pushing out business logic with little concern for html, MVC is all about clearing the way for uninhibited html.  The MVC View is all about http.  Scott said MVC values http.  Web forms values the control model, the abstraction.  This type of abstraction simply does not exist in the MVC world.  MVC is the place for people who like http and javascript.  I am not naturally one of those people and this helped me understand both my own struggle with and the power of MVC, and I think even for me there are important reasons for pursung this model despite the sudden annoyance I feel being remined that I am pretty much a JSON dummy, one of which is testability and separation of responsibilities within the framework.

 To see what I mean, just view the source of a big fat asp.net aspx page and compare that to the source from a nice looking asp.net MVC page.  The latter will be less complicated and is soweb 2.0 and a lot like the view page you built ot begin with.  No big ugly viewstate block; gone are the complex form ids with $ signs and compound structures.  It’s all about the html.  The truth is, those developers who don’t like messing with html will not be at home with MVC. 

This was a revelation for me.  I am not great with server side code but I understand that asp.net web forms pushed out too much html that was too complicated and too far from markup.  Our developers had to do back flips to get unit testing to work in the web forms context.  Let’s get clean http results and still have our choice of complex data models whether they be plugged into that old AS400 or the cloud platforms across the universe.