Last month marked the 40th anniversary of my having worked with Revelation Software products, so I'm going to indulge myself with a little wander down memory lane.

Cast your mind back to 1983 - the IBM PC had just been released in all its twin-floppied glory. For personal reasons I had to relocate away from home, and I was determined to find a job with a company car. An advertisement for a new company called IDM (with a logo that was suspiciously close to another Three Letter Acronym company) offered a sales position with a new software product just imported from the US - Revelation C. Their main business was selling Pick on IBM Series/1 and they had taken on Revelation as an afterthought due to the built in Terminal Emulator which allowed you to use your PC as an Adds Regent 100 emulator to any Pick based system. In those days hardware was EXPENSIVE, and a dumb terminal actually cost more to buy than buying Revelation. So, if you happened to have a PC and needed a terminal, buying Revelation was a no-brainer.

The early days were heady as everybody was scrambling to implement standalone solutions using this new PC and the market flooded with database programs. Companies made inflated claims for their products (I'm looking at you dBase II - you were NEVER relational despite what you claimed at the time) but Rev just powered ahead with a series of well thought out adverts that attracted likeminded developers.

Revelation D was an incremental development, but Revelation E blew our socks off. The guys from Cosmos came over to the UK to explain this new-fangled technology called "Networking", which Rev E supported. Now applications could scale to departmental level and with the scaling came an increasingly diverse client base. New features were added, and Rev F and Rev G made their way into the world. It was whilst working on a banking system written in Rev G that I got to work with an assistant who'd just graduated from Harvard. I'd complained that I needed a Business Analyst, and I got Jeff Bezos. But that's another story. (Spoiler - we didn't see eye to eye).

The other database companies started putting out more attractive development environments and Cosmos struggled to keep up, until one of their aspiring Sales People took it upon himself to rewrite Rev and thus was born Advanced Revelation (AREV). Microsoft's SQL Server was flying out of the door as people sought to build larger applications, and the fact that AREV had a SQL Bond that allowed rapid app development, made it a prime candidate for expansion. Business boomed and at one stage Revelation won an award from Microsoft for selling more SQL Server licenses than anybody else that year.

At the time the company I worked with split its database sales between Revelation and Microsoft SQL Server. As a rule of thumb, we told clients that if they had more than 50,000 rows in their main data table, they should look at SQL over Revelation. How I giggle looking back at that when lots of our clients these days have tables with tens of millions of rows.

During the next couple of decades as AREV matured, I got to work on some wonderfully varied systems. Seconded to HMRC I wrote some fascinating tax investigation software. There didn't seem to be a Government department who DIDN'T use Revelation, so I spent time at the Lord Chancellor's Office, the Home Office, MAFF, MOD et al. We even NEARLY came first in a Database Development Competition organised by a UK Computer journal. I say "nearly" because at the close of the practical part of the competition, the judges came to our stand and congratulated us on winning. It came as a surprise to both us and them, when a completely different company was announced as the winner. Though in fairness WE weren't taking out monthly full colour adverts in said Computer Journal.

When Sprezzatura launched in 1989 we started to collect an even more diverse client base, and the publication of REVMEDIA was a delightful 4 years where I got to work with some of the most interesting developers on the planet. I'd sit up until the early hours of the morning repeating experiment after experiment to see what changing Window Common actually did. I'd investigate issues brought up by subscribers and I'd try and work out compiled code structure to make working out what routines did just that little bit easier.

Another milestone during this time, was our involvement in the very first prosecution under the new Computer Misuse Act. We were occasionally consulting to a marketing company who employed their own in-house consultant. They decided that they no longer needed his services and so they let him go. Several months later their system began to behave strangely. Important data was suddenly meaningless gibberish - it had been encrypted. We were called in to investigate and what we found actually seriously impressed me. Putting together the pieces we established that the erstwhile former employee, knowing his time was limited, had placed an MFS on the main data table that, as it wrote to disk encrypted the data, decrypting it on reads, until the trigger date came when the MFS removed itself from the table and consequently stopped decrypting. We were able to show this to be the case and were called as Expert Witnesses for the upcoming trial at the Old Bailey. Turning up suited and booted we were actually quite disappointed that the gentleman in question pled guilty before the trial started. 

But then talk turned to OpenInsight (OI) and the AREV world went into limbo for a few years. Initial attempts to deliver a Windows product were wildly flawed. Reasoning that middleware was the way forward, Revelation released a Database Engine (oengine.exe) with Linear Hash support and blithely suggested that the established user base go away and learn something like Visual Basic to write their front end in.

There was rebellion and shortly thereafter OpenInsight 2 was born with the rudiments of what we know and love today.  It would be fair to describe it as a little flaky. Revelation invited us to partner with them in competing in an internationally famous Database Challenge. There were hundreds of entrants, but we were quietly confident of our ability to deliver. The challenge was to build a database application to a provided specification over the course of 8 hours. 5 hours in, disaster struck. The Repository became corrupted, and all of our work was lost. Knowing what we do now we could have fixed this, but this was very early days. So, with three hours to go for the challenge we had to start again. Regretfully this meant that we only came in the Top 20. If you ever wondered why the system USED to keep a backup duplicate of the Repository, that is why!

More years passed and Windows moved to 32 Bit. Once again Revelation was stuck in the mud. The 32 bit replacement was to be a similar rehash to OpenInsight 1 requiring the developer base to acquire Java skills - something many were reluctant to do. But after years of stagnation, Revelation passed into the hands of long term developer, Mike Ruane, who set about delivering a 32 bit version.

Shortly after commencing his tenure, he delivered on his promises and a fun few weeks were spent flying around the world with Mike and the crew showing the next 32 bit OI to an excited client base.

Once again, we got to work with great software, and to deliver some great technologies. In one particularly memorable assignment the Sprezz team rocked up on our courier client's premises and in two weeks delivered a fully functional on line booking system that blew the socks off the competition. In another, using S/Web we managed to create web sites processing millions of discrete transactions a day while the big database boys were boasting about their hundreds of thousands.

And now here we are with a completely rewritten OI, a thing of beauty in my mind and I'm glad I'm here to see it! We still get to work with a diverse group of people around the world, though sadly, some of our long-term colleagues are no longer with us. The core team of Sprezzatura delights me with our complementary skill sets and I can honestly not think of a group of people I'd enjoy working with more.

Over the past 40 years I have been privileged to work with Rev developers in Albania, Australia, Belgium, Canada, the Caribbean, Czechoslovakia, Denmark, England, Finland, France, Germany, Ghana, Hong Kong, India, Ireland, the Netherlands, New Zealand, Norway, Portugal, Puerto Rico, Saudi Arabia, Scotland, South Africa, Sweden, Switzerland, Turkey, Venezuela, Wales and of course in a large number of States of the good ol’ US of A. I’ve met some wonderful people and some of those have even changed my life in very positive ways outside of computing!

I count my blessings that 40 years ago I decided to put my money on the Revelation horse. I had no clue what I was doing at the time, but it has served me incredibly well. There aren't many people in IT who can say that they've not had to learn a new product since starting their career :). And it's a testament to both the product and ourselves that we still have clients we started work with back in 1984! We're all a bit older, hopefully a bit wiser, but no less enthusiastic.

So, in conclusion, if you're reading this, it's likely you're part of the Rev community and it's a pleasure to be in it with you. Here's to the next decade! Cheers!

 On one of our more recent projects, we were required to display a PDF document by double-clicking in an edit table. No real issue, we can use just use SHELLEXEC as documented here right?

Wrong.

The problem with a SHELLEXEC, is that by default, it launches a browser that has an address bar  containing the full URL to the document in question, allowing the user to modify the URL and potentially access information they were not privy to. (The users don't have browse rights so there isn't a problem of them browsing to other information, but knowing the fully formed URL they could take intelligent guesses at other potential document names). So, we considered looking at turning the address bar off, but then remembered the new WebView control. A control that allows you to embed the Microsoft WebView2 Edge Browser directly into an OpenInsight form and to interact with it.

This looked like it was going to be the easiest hack ever. Create a form with just two controls, the WebView control and a close button. Have it accept as a create parameter the document to display, and all is good, yes?

So we added a quickevent to the form that simply set the URI property of the WebView control, to the value passed in the createParam for the form. 

It ran.

It did nothing.

Puzzled we modified the CREATE quickevent to also update the screen caption with the file name we were trying to display. (This ability to stack quickevents rocks!). 

The caption changed to the correct value but still nada.

Thinking laterally we invoked the system monitor and used the SP command to set the URI of the control to the value we had passed in. 

It displayed.

Rinse and repeat.

Chatting to the developers, it seems the issue here is that the WebView control takes a little time to instantiate itself, so it it might not be finished before the CREATE event runs. So we have to wait for the WebView control to be finished and THEN set the URI.

Sounds like a case for looping, querying status and waiting, yes?

No.

The WebView control is asynchronous. So all we have to do is wait for an event that tells us that the WebView control is ready, and THEN set the URI.

All this took was two quickevents. 

One on the CREATE event, setting a synthetic property to contain the document name

and one on the WEBVIEWCREATED event for the WebView control (which is fired when the WebView is fully instantiated) which retrieves the synthetic property and uses it to set the URI property.

 Occasionally RTI slip out little tools and utilities that they've found useful. If you're very lucky they'll even ship source. One such gem in OI 10 is the OIWIN Date/Time picker - RTI_DATETIME_POPUP to its friends.

If ever you've wanted to offer your user a way of selecting date or time from a calendar/time control, and you've not been too happy with the datetimepicker OCX and the old POPUP_MONTH dialog seems a little dated, then this is the one for you.

It really is stunningly simple to use. Just add a quickevent to call a stored procedure and you're sorted.

For illustrative purposes, here's a sample window -

The cuebanners reflect the conversion on the edit line. The quickevent for the options on each editline is defined thusly - 

Once you've chosen the Target you can just accept the defaults other than DEFPROP which should be replaced with INVALUE.

We only need one options button because we're taking advantage of a newly exposed property - in the property panel under the Behavior section, we're setting ALLOWFOCUS to false and then we can just send an OPTIONS event to @FOCUS.

So how does the routine behave? See the following screen shots -

Now there are a couple of caveats that I need to mention. The first (as you can see) is that DT and DTS provide the same dialog. The second (due to a bug that I leave you to find) is that DTS always returns 1967 as the year. But as you are encouraged to save under a different name and compile your own version it is easily fixed! If you don't want to then it is all fixed with the latest beta.

The examples given below show what displays when you let the system determine default values. Examining the source reveals that, should you wish to, you may have much more granular control over the behaviour of the control. In this endeavour RTI_POPUP_DATETIME_EQUATES are your friend, but to save you having to look at this, we'll document their use here.

If you use DIALOG_BOX to launch the RTI_POPUP_DATETIME window, you can pass in an @Fm delimited create parameter having the following values -

    <1>    Mode, D, DT, DTS, MT, MTS
    <2>    Initial date to use in internal format
    <3>    Initial time to use in internal format
    <4>    Parent
    <5>    Horizontal alignment against owner (L or R)
    <6>    Value mark delimited array containing the screen size of the owner
    <7>    First year in the year dropdown
    <8>    Last year in the year dropdown

We can override date ranges, set default values and even alter dialog positioning by passing in the correct values.

One of the biggest differences between OI TCL and AREV TCL is that the latter made use of the VOC file to allow the user to customise commands and even to add their own commands. This does not seem to be the case in OI, rather a cursory examination of the object code for RTI_IDE_TCL suggests that the command list is hard coded. 

With this in mind it might be useful to document exactly what commands are supported, and where unclear, what they actually do. For the purposes of this article it will be assumed that the reader is familiar with the syntax of AREV TCL commands. When optional variables are nominated they will normally be in the order TABLE ROW(S) OPTIONS.

So in alphabetical order, the commands are as follows - 

 Recently we were stumped by a compilation error when using pipes to create a multi line statement, this sort of thing,

    if value then

        on inlist("ALPHA,BRAVO,CHARLIE", value, ",") goSub |

            doAlpha, |

            doBravo, |

            doCharlie

    end

the compiler just kept complaining about an unmatched then/end else. We thought that the only way to get to the bottom of this was to look at what the precompiler produced to see where the error was. Struggling we tried to recall the syntax of the pragma to save the precompiled output. 

Fortunately a more knowledgeable team member pointed out that using the pipe stripped spaces by design, to allow aesthetic alignments of strings. What this meant was that the compiler was not compiling goSub doAlpha, but rather, goSubdoAlpha.

The solution, repeating pipes. Using two pipes prevents the spaces being stripped and 

    if value then

        on inlist("ALPHA,BRAVO,CHARLIE", value, ",") goSub ||

            doAlpha, |

            doBravo, |

            doCharlie

    end

compiled just fine. 

Anyway it's been 16 years since our article on using pragmas and it's still as relevant as ever, so why not check it out and refresh your memory!

We recently came across a situation that we have literally never encountered in our decades of working with Linear Hash. Basically a FIXLH operation was interrupted mid process and hi-jinks ensued. The resultant mess took a while to sort out but our findings may prove useful if you ever find yourself in a similar situation.

This specific issue was encountered on an AREV 3.12 system but the logic for FIXLH remains roughly the same so the information remains applicable.

Basically, the FIXLH operation was interrupted about 10% of the way through processing (the NTVDM abended). A quick check of the table header revealed that it believed itself to be a clean empty table with no rows. Yet listing the table produced a set of row ids after an initial pause while nothing was apparently happening.

What had happened was that the table header had been reset to that of an empty table, and the first 10% or so of the primary frames had been blanked. This caused us to investigate the processing undertaken by FIXLH and we can confirm that it is as follows:

    Create and/or clear the temporary dumpfix table

    grab frame zero group

    write rows in group to temporary dumpfix table

    set frame zero header to that of an empty table

    loop

        grab next group

        write rows in group to temporary dumpfix table

        wipe primary frame to null

    until no groups left

    repeat

    copy rows in temporary table back to source table

So if the operation is interrupted, some of the data will be in the source table and some will be in the temporary dump fix table. 

As an additional caveat, note that if the rows in the temporary dump fix table are not stored off elsewhere, they will be lost next time FIXLH is run.

Since the advent of the UD we've never had to actually fix a GFE for quite some time, so this was an eye-opening operation!

Backups are your friend...

Recently at a client site, we encountered an error that was a little more baffling than normal.  Under very specific circumstances,    RLIST would bail with no results.  Eventually, we had a duplicatable test sentence which was basically

    SELECT TABLE BY CALC_COL_1 WITH REAL_FIELD_1 = ‘XXX’ AND WITH CALC_COL_2 = ‘YYY’

When we removed the BY clause (CALC_COL_1) our sentence ran to completion.   This put the problem squarely on CALC_COL_1, which is where we started debugging.  

We changed CALC_COL_1 to a simple “@ANS=1” and ran the sentence again.  This worked correctly.  To prove to ourselves that CALC_COL_1 was the problem, we put back the original code and changed our sentence to 

    SELECT TABLE BY CALC_COL_1 WITH REAL_FIELD_1 = ‘XXX’ AND WITH CALC_COL_3= ‘ZZZ’

This also worked.   

Now we’re starting to get a little confused.  Modifying CALC_COL_1 solved the problem.  Replacing CALC_COL_2 also solved the problem.  It has to be some sort of interaction with the calculated columns, because there’s no obvious consistency lining up the errors with the sentence structure.

It’s time to break out the profile log and see if something jumps out, and it did.  Something appeared off with EXTERNAL.SORT.  It seems like it executed a few times, and then everything stopped.  We dusted off our copy of PROCMON.EXE ¹ and ran the process again.  PROCMON showed that the sort file was being opened, but that it was not being closed.  It also showed a lot of file access  We’re now convinced this has something to do with sorting in general, and not with the specific field being sorted.

After trying well over a dozen different variations on the RLIST sentences, we finally worked out that we needed two things to happen for the error to occur.  The first was that we had to have CALC_COL_1 as the sort column.  The second was that the WITH clauses needed to return enough keys to force the system into using EXTERNAL.SORT instead of a simple in-memory sort.

Normally, when OpenInsight starts behaving very oddly, the culprit is usually a function that’s not clearing set_status() before calling get_status.  When that happens, the get_status() call could be returned an error from 10 programs back.  We reworked the calculated columns to ensure we clear set_status, and feeling very proud of ourselves, sat back and waited to bask in the glory offered to us by our client.  Obviously, the report failed again.

Sitting back, we start wondering if maybe there’s a small part of the RLIST code that’s still thinks it’s Advanced Revelation and it’s actually checking the pseudo STATUS() variable.  So, we reworked the calculated columns to ensure we clear set_status and status(), and feeling very proud of ourselves, sat back and waited to bask in the glory offered to us by our client.  Obviously, the report failed yet again.
 
Surely there can’t be three ways to set an error condition in OpenInsight?  It turns out that there is.  We suddenly remembered @FILE.ERROR.  So, we rework the calculated columns to ensure we clear set_status and status() and @FILE.ERROR, and feeling very proud of ourselves, sat back and waited to bask in the glory offered to us by our client.  And this time there was much rejoicing and basking.

With the solution in hand, we set out to find out exactly why this happened and where the system was registering the failure.  What we found was the calculated field was doing a lot of file access and would attempt to access records that didn’t exist.  This was what was setting @FILE.ERROR.  But, we hit a two-fer, as they say.  Some of the randomness had to do with the amount of data returned.  If enough sort data was returned the system would call EXTERNAL.SORT to sort the data.  EXTERNAL.SORT writes to temporary OS files, and the OS file commands use @FILE.ERROR to check for errors.  EXTERNAL.SORT would execute an OS operation, check @FILE.ERROR, find the pre-existing error, and abort the process.  But, this would only happen if the very last row processed set @FILE.ERROR.  Otherwise RTP57 would clear any prior @FILE.ERROR settings, and @FILE.ERROR would be null during the EXTERNAL.SORT call. 

1.  PROCMON and other useful tools are available as part of the Sysinternals suite of development tools, which no programmer should be without.