Just before Christmas we were invited by one of our clients to visit their site and to perform a number of OI related tasks in a very finite time frame. Specifically to upgrade a 16 bit OI 3.7.5 production system and an OI 8.0 web based system to OI 9.2+ whilst installing the UD Heavy. We were allocated 5 days to do this so were confident that it was achievable.

Of course our confidence was rooted in real world experience - both of of our combined skill sets and of typical UK weather...

So early Friday afternoon came and exercising caution the Sprezz lead consultant ("the Sprezzie") on this project began the 200 mile drive to the client - this was to be a Sat/Sun/Mon/Tue/Wed job to minimise impact on the factory production system and to fit in with some tight scheduling requirements for Sprezz. Unbeknownst to us this was to mark the beginning of some of the worst weather in recent UK history. In a normal world this 200 mile journey would take 3.5 hours.

Two hours into the journey the Sat Nav decided to change the route due to traffic conditions ahead and rerouted the car to within 20 miles of the original starting point then started again... snow was blizzarding and road conditions were becoming treacherous. Average speeds dropped and past experience of snow driving became essential. The next 5 hours or so were not fun but eventually the Sprezzie arrived at their target hostelry - a snowed in Bed and Breakfast in the middle of nowhere - just in time to discover that after a 7.5 hour journey, all the places serving food had shut. And so to bed.

The next morning dawned and the 12 mile drive to the client had never looked so difficult. The roads were snowed in and the gritting lorries had yet to visit. Thanking providence for front wheel drive the Sprezzie eventually got to the client and started work. The initial UD Heavy installation was a cinch, it installed and it did what it said on the tin. The 4.6 UDH is one we at Sprezz Towers have a lot of experience with and we have no hesitation in recommending it to our clients. It addresses a lot of issues found in previous releases of the UDH and the documentation is an order of magnitude better!

The 9.2 upgrade was a bit more of an issue - the install was fine but combining the 3.7.5 and the 8.0 system proved to be a bit more of a challenge. Eventually two APPBACKUPs were taken and a custom program was created in conjunction with the client to combine "the best of both worlds" and this custom program installed all elements into a virgin 9.2 system without the need for an APPRESTORE.

Everything came together BUT the system wasn't running for some reason. So thus began the multi-sourcing quest. First off we downloaded Sprezz's "Recompile All" utility. This recompiled all events, stored procedures and windows. Things started to run again but the windows looked ugly. So then we downloaded Rev's "Font changer" utility and changed all the fonts to Tahoma in Windows and Popups. Finally we downloaded and ran SRP's form fixer to reset the errant style bits and we were back in the running.

Next up the client agreed that hiding the LK and OV files was a worthy goal - no more users accidentally trashing systems. So we followed the incredibly simple instructions in the UDH installation guide and attempted to hide the files. It should be said that the installation was on a 64 bit server. The Sprezzie in charge tried and tried but could not get the files to hide - the registry settings would not take. So throwing it open to all Sprezz crew he mailed the internal distribution list and was cajoled by colleagues into triple checking his registry entries. To much embarrassment it became apparent that Rev create two key sets in the WOW3264 section - one as Revelation Software for the client - and one as RevSoft for the UDH. It behoves the developer to use the RevSoft key if s/he wants Shares to work...

So onto the next thing on Santa's wish list - converting their OECGI installation to OECGI3 on an IIS 7 web server. Quick start read, product up and running in less than an hour. Way to go RevSoft! These quick start guides REALLY help.

Starting to wrap up and wind down the client mentions that they'd like to convert their existing mail server (all based around Rev's MAPI technology) to a more future proof technology. No problem - Sprezz's free SMTP mail client is downloaded and installed and the conversion of the mail server continues apace - until.... what? You want to CONSUME mail as well as PRODUCE it? Uh Oh....

It suddenly became apparent that there is a gap in the RevSoft offerings...

GETMAIL as shipped with OI is a handy socket based POP3 mail client which pulls down the mail from the pop server and delivers it as text files to the client - the only downside is that it

a) is underdocumented and
b) it delivers effectively binary streams unlike the MAPI equivalent that delivers nicely documented structures.

Fortunately for us (and the client) the client only needed to consume several very specific mail formats so again mining the collective Sprezz gestalt we were reminded of the existence of BASE64DECODE in OI and were able to put together an OI based program to decode multi part MIME emails in a manner consistent with the existing MAPI calls. This allowed the existing Mail Server technology to be migrated to SMTP/POP3 with minimal change.

All in all a fun filled and technically challenging week made possible by the flexiblity of the OI product, the contributions of the Rev community and the whole being greater than the parts that is the Sprezz consultancy team.

Labels: 9.2, BASE64DECODE, GETMAIL, MIME, UDH

It has come to our attention recently about a misconception regarding the lore of the Universal Driver. It has been commonly accepted by all (including us here at Sprezz Towers) that on small networks (2-4 users), using one of the Universal Drivers without an NT Service is the preferred network configuration.

However, after a lengthy debugging session involving OpenInsight 9.1.1 and the UD 4.6, Revelation US has verified that without a Linear Hash Server product, you must use the All Networks Driver.

To quote Revelation:


With the Universal Driver NUL (Network User License) free for all OpenInsight 9.x customers, we urge all pure OpenInsight shops not using a Linear Hash service to install the Universal Driver 4.6 NUL on your servers.

Those customer that choose not to upgrade to 9.x and are not using a Linear Hash Service product should switch drivers to the All Networks as soon as possible. Remember, switching back to the All Networks driver could cause you problems if any of your tables have been created with the new UD style header.

To help with this, Sprezzatura has enhanced our Revert LH Utility. This takes a linear hash table with the new UD 3 style header and converts it to standard LH format, readable by all network drivers.

Since the last release a few years back, we've fixed a few small bugs involving keys traversing frame boundaries. We've also added a new user interface window to help you with reverting your files back to the original header structure.


Our new user interface allows you to see the Revelation file names of the files you wish to revert. It also allows you to pick and choose files, if, for some reason, you'd like to keep the new header on some files.

This is a free of charge utility, and can be obtained by contacting us here.

Please note: The free version of the UD 4.6 does not work with any version of OpenInsight prior to version 9. It also does not work with Advanced Revelation.

(This post brought to you by the letter A(aron))

Labels: Linear Hash, UDH, Universal Driver

We've got a couple of clients with UDH installations and this weekend we upgraded one of them to 4.6 - the latest and greatest. There's so much to like about this release - not least the improved documentation and improved replay of journalling. But sometimes the unforeseen happens and you hit a "Well nobody would ever do that would they?" moment - the moment when just as you ask yourself the question you see several possible scenarios unfold where not only might they do that, they'd have damn good reasons for doing it.

So after the weekend was over we returned to base and expected a tranquil week. Until Monday morning that is. We seemed to have two unrelated problems :-

• Users were getting kicked out with "Maximum number of variables" in RTP32
• The Event Viewer on the (Windows 2008 r2) UDH Server was full of thousands of similar error messages.

Well fixing the RTP32 problem was going to be easy - We just consulted our on-line REVMEDIA for what RTP32 actually does.  OK FIELDSTORE. So we'd just look for where that was going into an infinite loop. For details of how we'd actually achieve this why not come to our talk on using logs to troubleshoot Revelation problems at RevCon 2010?

Regretfully it didn't seem to be as easy as that as we weren't actually making much use of FIELDSTORE. So we'd come back to that issue and concentrate on the Event Viewer issue. As we said the Event Viewer on the (Windows 2008 r2) UDH Server was full of thousands of similar error messages. All of which made no sense whatsoever to us. To make matters worse, scattered amongst these errors were more meaningful ones.

Thankfully as ever the tech team at RevSoft swang into action and decided that we was seeing two problems. One they thought might be related to messages (we'll come back to that) and one that might indicate that we did in fact have data errors in some of our tables.

Tracking down the data errors was made difficult by the fact that with nearly 10,000 errors to choose from it's difficult to search for a specific message - in this case the text "REV" to find the actual DOS files affected. Initial attempts to export from the Event Viewer failed as selecting Tab delimited text resulted in a small subset of the log being exported. After several failed attempts we dropped back to CSV and lo and behold, a 70K file became a 12 MB file. With this we were able to open in notepad and identify the files in question. One was FUBAR, a 10K LK and a 4 GB OV - on a dictionary file. That was soon rectified. The second was a null key which caused the UDH to report a LHReadNextKeyIdGroup error with an unfeasibly large error code.

Now we had the physical errors sorted it was time to address the logical. Revelation suggested that what we might be seeing could be a problem with overly long messages. 


We were slightly at a loss as to why this might be until they pointed out that in AREV when you call MSG it firstly assumes that the first passed parameter is a row id and tries first to read it from MESSAGES and then SYSMESSAGES before deciding that it was a literal and attempting to display it. The reason this becomes a problem is that in 4.6 maximum row id lengths are enforced and generate errors if exceeded.

We set about creating  proof of this hypothesis and wrote a program to display an increasingly long message.


and sure enough we dropped to the debugger


in RTP32! Hurrah! And what was the magic number?


553 - one more than the maximum allowed key length of 552.

So now we had the problem the solution became obvious (well talking it over with colleagues helped - "by the power of Sprezz"....) - we'd just write a shell program to intercept message calls and if they were longer than the longest message key we'd pass over a message map that told MSG that what was being passed was a literal so it didn't need to try reading it. So $MSG was copied to $MSG_RTI and our replacement was created. And as you can see it worked...


We implemented and all seemed to be going swimmingly but just when we thought it was safe to go back into the water we started getting a new batch of errors which seemed to be coming from input validation not direct message calls. Fortunately now that we had our own message shell it was easier to check the program stack and find out the culprit...

It seems that when the user was entering an invalid key there were a number of possible patterns the key could match. So IN.PATTERN was called to see which, if any it passed. On determining that the pattern match had failed PAT_CHECK was called to construct an "English type" failure message. This message was passed to MSG as a msgRec with msg being left blank. So once again MSG_RTI looked at the script and tried to read it as a message - failing as before. So once again all we had to do was check any passed script and if it was longer than our maximum row id length set the literal flag. Once these modifications were made all that was left were legitimate day to day errors.

We present below the code we used which you are free to use at your own risk - Sprezzatura makes no warranties etc etc etc. If you want warranties we'll happily come and help you for our usual daily rate!

Labels: UDH

We've had some good successes with the Universal Driver Heavy, including my own personal favourite - installing it on an AREV 2.12 site despite the warnings on the tin! It took a bit of tweaking but we got there.

Anyways on two separate sites recently we needed to replay the journal files to bring the secondary server back up to synchronicity with the primary server and on both of these sites the process would fail, resulting in us having to copy the secondary from the primary - thus rather removing the point of having the UDH in the first place.

We'd start the UDH and launch the manager to instruct the service to go into mirroring and it would happily start chugging along doing something - perfmon showed a lot of I/O and CPU activity associated with the LH31SRVC.EXE so we were confident that it was trying to do something, but after a short while the LH Manager would start to display REV_LOADREC errors and crash. Now we well knew that this meant that the LH Manager could not contact the service and sure enough the service was no longer in memory but it had exited so cleanly that there were no reports of errors in the UD Log OR the Event Log.

Given the complete lack of available diagnostic information we were initially stumped until the idea of running the service in debug mode was mooted. So opening a command prompt we changed to the UDH directory and executed LH31SRVC.EXE -debug.

The service started reading through the journal files and optmising them. The numbers crept up, 10, 20, 30, 40 and then at 44 the UDH just abended. Well, exited is probably more accurate as there were no errors to speak of. It would seem that something in the 44th file was causing problems for the UDH.

Revelation are understandably keen to ensure that the UDH is as stable as possible so the files were mailed to Revelation who tested them on the latest 4.6 build of the UDH (after writing a utility to make the journal files the correct format) and discovered the same error. At this point they lept into sleuth mode and within hours were able to declare that the UDH replay had some fairly fatal problems when row id length exceeded 512 bytes!

Now given that the maxiumum row id size has been documented as 50 characters we were surprised by this strangely excessive row id length, but the fix was confirmed and in future releases of the LH driver a maximum row id length of 512 bytes will be enforced - attempts to write with a longer id will cause the ELSE branch of the write to trigger.

Mind you these investigations did trigger an interesting discovery. Given that the maximum row id size was 50 bytes, the system verify routines would report a GFE if any such row ids were encountered. So if you've ever had a mysterious GFE which didn't seem to be there then check the size of your row ids! We had to develop a routine to do such a thing but that's a subject for another blog post!

Labels: UDH, Universal Driver Heavy