Home page Home page Home page Home page
Pixel
Pixel Header R1 C1 Pixel
Pixel Header R2 C1 Pixel
Pixel Header R3 C1 Pixel
Pixel
By Sprezz | Wednesday 7 February 2024 16:32 | 0 Comments

Recently we were playing with a routine to recurse through directory structures on a nominated disk and attempt to osOpen a specific file at a location - a quick utility to hunt down rogue OpenInsight installations by looking for SYSPROG DBTs. What started off as a vanity project very quickly became a learning exercise about how the system handles OS files behind the scenes.

The routine was working fine, correctly identifying locations containing OpenInsight but then we tried pointing it at Drive C. Suddenly the routine was finding rogue OpenInsight installations all over the disk, SYSPROG.DBTs were opening like they were a dime a dozen. Rogue installation-a-rama.

Except they weren't actually there. OsOpen claimed they were there - well it reported that it had successfully opened the file - but on closer inspection there was no file actually there.

There seemed to be no great mystery about what the apparent rogue installations had in common - they all had humongously long folder paths. Once the path gets beyond a certain length, attempts to open files that aren't there succeed. Rather than keep you in suspense we'll just confirm that this length is a common Windows constant called MAX_PATH - which if you check your msWin_MAX_PATH_Equates insert (yes - there is one) equates to 260 characters.

We reported this to those good folks at Revelation and further inspection revealed that, yes, there is an issue when opening files whose locations are longer than 260 characters. It was explained that internally the system stores the OS handle in an array of up to 128 values and that these values were predeclared at MAX_PATH plus a bit. 

This interested us, because we never realised that there was a limit of 128 concurrently open OS files in the Rev product line. We'd always assumed that (given that the osOpen file handle is just the name of the file) the system opened and closed the file each time it was written to or read from. That assumption was incorrect as, as explained above, Rev keep an array of OS file handles. They take the osOpen handle, look it up in an array and extract the OS file handle. 

This got us thinking, because we aren't always diligent about osClosing handles. Why had we never seen an issue with over 128 files? So we set about testing our assumptions.

It transpires that, if the system uses up the 128 handles, it starts to reuse the array slots in a FIFO manner. So if you subsequently try an OS operation on a handle that has been dropped, Rev reopens it and drops another. 

The implication of this (which we tested to be true) is that osOpen is not actually needed for OS operations.

We don't recommend NOT using it we hasten to add, this was just an academic exercise which could be invalidated at any time. 

As an aside when Rev gets the OS handle it places a lock on the file in question. You can see this if you attempt to access the file from outside of OpenInsight. You'll see a message like this -



The lock is removed when either you osClose the file or the system does so during its FIFO operation. A good tip from Rev is that, and I quote

"when logging to a file I like to use

osOpen myFile to myHandle

offset = dir(myfile)<1>

oswrite myChunk on myHandle at offset

osClose myHandle

This avoids holding the lock, so you can use notepad++ to look at the changes while a process is running.".

Anyway you'll be pleased to know that for subsequent releases of OpenInsight the restriction on long path names has been removed - along with the 128 element array limitation. 

By Sprezz | Wednesday 4 October 2023 10:00 | 10 Comments

Last month marked the 40th anniversary of my having worked with Revelation Software products, so I'm going to indulge myself with a little wander down memory lane.

Cast your mind back to 1983 - the IBM PC had just been released in all its twin-floppied glory. For personal reasons I had to relocate away from home, and I was determined to find a job with a company car. An advertisement for a new company called IDM (with a logo that was suspiciously close to another Three Letter Acronym company) offered a sales position with a new software product just imported from the US - Revelation C. Their main business was selling Pick on IBM Series/1 and they had taken on Revelation as an afterthought due to the built in Terminal Emulator which allowed you to use your PC as an Adds Regent 100 emulator to any Pick based system. In those days hardware was EXPENSIVE, and a dumb terminal actually cost more to buy than buying Revelation. So, if you happened to have a PC and needed a terminal, buying Revelation was a no-brainer.

The early days were heady as everybody was scrambling to implement standalone solutions using this new PC and the market flooded with database programs. Companies made inflated claims for their products (I'm looking at you dBase II - you were NEVER relational despite what you claimed at the time) but Rev just powered ahead with a series of well thought out adverts that attracted likeminded developers.

Revelation D was an incremental development, but Revelation E blew our socks off. The guys from Cosmos came over to the UK to explain this new-fangled technology called "Networking", which Rev E supported. Now applications could scale to departmental level and with the scaling came an increasingly diverse client base. New features were added, and Rev F and Rev G made their way into the world. It was whilst working on a banking system written in Rev G that I got to work with an assistant who'd just graduated from Harvard. I'd complained that I needed a Business Analyst, and I got Jeff Bezos. But that's another story. (Spoiler - we didn't see eye to eye).

The other database companies started putting out more attractive development environments and Cosmos struggled to keep up, until one of their aspiring Sales People took it upon himself to rewrite Rev and thus was born Advanced Revelation (AREV). Microsoft's SQL Server was flying out of the door as people sought to build larger applications, and the fact that AREV had a SQL Bond that allowed rapid app development, made it a prime candidate for expansion. Business boomed and at one stage Revelation won an award from Microsoft for selling more SQL Server licenses than anybody else that year.

At the time the company I worked with split its database sales between Revelation and Microsoft SQL Server. As a rule of thumb, we told clients that if they had more than 50,000 rows in their main data table, they should look at SQL over Revelation. How I giggle looking back at that when lots of our clients these days have tables with tens of millions of rows.

During the next couple of decades as AREV matured, I got to work on some wonderfully varied systems. Seconded to HMRC I wrote some fascinating tax investigation software. There didn't seem to be a Government department who DIDN'T use Revelation, so I spent time at the Lord Chancellor's Office, the Home Office, MAFF, MOD et al. We even NEARLY came first in a Database Development Competition organised by a UK Computer journal. I say "nearly" because at the close of the practical part of the competition, the judges came to our stand and congratulated us on winning. It came as a surprise to both us and them, when a completely different company was announced as the winner. Though in fairness WE weren't taking out monthly full colour adverts in said Computer Journal.

When Sprezzatura launched in 1989 we started to collect an even more diverse client base, and the publication of REVMEDIA was a delightful 4 years where I got to work with some of the most interesting developers on the planet. I'd sit up until the early hours of the morning repeating experiment after experiment to see what changing Window Common actually did. I'd investigate issues brought up by subscribers and I'd try and work out compiled code structure to make working out what routines did just that little bit easier.

Another milestone during this time, was our involvement in the very first prosecution under the new Computer Misuse Act. We were occasionally consulting to a marketing company who employed their own in-house consultant. They decided that they no longer needed his services and so they let him go. Several months later their system began to behave strangely. Important data was suddenly meaningless gibberish - it had been encrypted. We were called in to investigate and what we found actually seriously impressed me. Putting together the pieces we established that the erstwhile former employee, knowing his time was limited, had placed an MFS on the main data table that, as it wrote to disk encrypted the data, decrypting it on reads, until the trigger date came when the MFS removed itself from the table and consequently stopped decrypting. We were able to show this to be the case and were called as Expert Witnesses for the upcoming trial at the Old Bailey. Turning up suited and booted we were actually quite disappointed that the gentleman in question pled guilty before the trial started. 

But then talk turned to OpenInsight (OI) and the AREV world went into limbo for a few years. Initial attempts to deliver a Windows product were wildly flawed. Reasoning that middleware was the way forward, Revelation released a Database Engine (oengine.exe) with Linear Hash support and blithely suggested that the established user base go away and learn something like Visual Basic to write their front end in.

There was rebellion and shortly thereafter OpenInsight 2 was born with the rudiments of what we know and love today.  It would be fair to describe it as a little flaky. Revelation invited us to partner with them in competing in an internationally famous Database Challenge. There were hundreds of entrants, but we were quietly confident of our ability to deliver. The challenge was to build a database application to a provided specification over the course of 8 hours. 5 hours in, disaster struck. The Repository became corrupted, and all of our work was lost. Knowing what we do now we could have fixed this, but this was very early days. So, with three hours to go for the challenge we had to start again. Regretfully this meant that we only came in the Top 20. If you ever wondered why the system USED to keep a backup duplicate of the Repository, that is why!

More years passed and Windows moved to 32 Bit. Once again Revelation was stuck in the mud. The 32 bit replacement was to be a similar rehash to OpenInsight 1 requiring the developer base to acquire Java skills - something many were reluctant to do. But after years of stagnation, Revelation passed into the hands of long term developer, Mike Ruane, who set about delivering a 32 bit version.

Shortly after commencing his tenure, he delivered on his promises and a fun few weeks were spent flying around the world with Mike and the crew showing the next 32 bit OI to an excited client base.

Once again, we got to work with great software, and to deliver some great technologies. In one particularly memorable assignment the Sprezz team rocked up on our courier client's premises and in two weeks delivered a fully functional on line booking system that blew the socks off the competition. In another, using S/Web we managed to create web sites processing millions of discrete transactions a day while the big database boys were boasting about their hundreds of thousands.

And now here we are with a completely rewritten OI, a thing of beauty in my mind and I'm glad I'm here to see it! We still get to work with a diverse group of people around the world, though sadly, some of our long-term colleagues are no longer with us. The core team of Sprezzatura delights me with our complementary skill sets and I can honestly not think of a group of people I'd enjoy working with more.

Over the past 40 years I have been privileged to work with Rev developers in Albania, Australia, Belgium, Canada, the Caribbean, Czechoslovakia, Denmark, England, Finland, France, Germany, Ghana, Hong Kong, India, Ireland, the Netherlands, New Zealand, Norway, Portugal, Puerto Rico, Saudi Arabia, Scotland, South Africa, Sweden, Switzerland, Turkey, Venezuela, Wales and of course in a large number of States of the good ol’ US of A. I’ve met some wonderful people and some of those have even changed my life in very positive ways outside of computing!

I count my blessings that 40 years ago I decided to put my money on the Revelation horse. I had no clue what I was doing at the time, but it has served me incredibly well. There aren't many people in IT who can say that they've not had to learn a new product since starting their career :). And it's a testament to both the product and ourselves that we still have clients we started work with back in 1984! We're all a bit older, hopefully a bit wiser, but no less enthusiastic.

So, in conclusion, if you're reading this, it's likely you're part of the Rev community and it's a pleasure to be in it with you. Here's to the next decade! Cheers!

By Sprezz | Tuesday 5 September 2023 12:57 | 2 Comments

 On one of our more recent projects, we were required to display a PDF document by double-clicking in an edit table. No real issue, we can use just use SHELLEXEC as documented here right?

Wrong.

The problem with a SHELLEXEC, is that by default, it launches a browser that has an address bar  containing the full URL to the document in question, allowing the user to modify the URL and potentially access information they were not privy to. (The users don't have browse rights so there isn't a problem of them browsing to other information, but knowing the fully formed URL they could take intelligent guesses at other potential document names). So, we considered looking at turning the address bar off, but then remembered the new WebView control. A control that allows you to embed the Microsoft WebView2 Edge Browser directly into an OpenInsight form and to interact with it.

This looked like it was going to be the easiest hack ever. Create a form with just two controls, the WebView control and a close button. Have it accept as a create parameter the document to display, and all is good, yes?

So we added a quickevent to the form that simply set the URI property of the WebView control, to the value passed in the createParam for the form. 

It ran.

It did nothing.

Puzzled we modified the CREATE quickevent to also update the screen caption with the file name we were trying to display. (This ability to stack quickevents rocks!). 

The caption changed to the correct value but still nada.

Thinking laterally we invoked the system monitor and used the SP command to set the URI of the control to the value we had passed in. 

It displayed.

Rinse and repeat.

Chatting to the developers, it seems the issue here is that the WebView control takes a little time to instantiate itself, so it it might not be finished before the CREATE event runs. So we have to wait for the WebView control to be finished and THEN set the URI.

Sounds like a case for looping, querying status and waiting, yes?

No.

The WebView control is asynchronous. So all we have to do is wait for an event that tells us that the WebView control is ready, and THEN set the URI.

All this took was two quickevents. 

One on the CREATE event, setting a synthetic property to contain the document name


and one on the WEBVIEWCREATED event for the WebView control (which is fired when the WebView is fully instantiated) which retrieves the synthetic property and uses it to set the URI property.




The results achieved what the client wanted. A displayed document with no identifying file paths. And no code. 
By Sprezz | Tuesday 1 August 2023 15:59 | 0 Comments

 Occasionally RTI slip out little tools and utilities that they've found useful. If you're very lucky they'll even ship source. One such gem in OI 10 is the OIWIN Date/Time picker - RTI_DATETIME_POPUP to its friends.

If ever you've wanted to offer your user a way of selecting date or time from a calendar/time control, and you've not been too happy with the datetimepicker OCX and the old POPUP_MONTH dialog seems a little dated, then this is the one for you.

It really is stunningly simple to use. Just add a quickevent to call a stored procedure and you're sorted.

For illustrative purposes, here's a sample window -


The cuebanners reflect the conversion on the edit line. The quickevent for the options on each editline is defined thusly - 




Once you've chosen the Target you can just accept the defaults other than DEFPROP which should be replaced with INVALUE.

We only need one options button because we're taking advantage of a newly exposed property - in the property panel under the Behavior section, we're setting ALLOWFOCUS to false and then we can just send an OPTIONS event to @FOCUS.





So how does the routine behave? See the following screen shots -






Now there are a couple of caveats that I need to mention. The first (as you can see) is that DT and DTS provide the same dialog. The second (due to a bug that I leave you to find) is that DTS always returns 1967 as the year. But as you are encouraged to save under a different name and compile your own version it is easily fixed! If you don't want to then it is all fixed with the latest beta.

The examples given below show what displays when you let the system determine default values. Examining the source reveals that, should you wish to, you may have much more granular control over the behaviour of the control. In this endeavour RTI_POPUP_DATETIME_EQUATES are your friend, but to save you having to look at this, we'll document their use here.

If you use DIALOG_BOX to launch the RTI_POPUP_DATETIME window, you can pass in an @Fm delimited create parameter having the following values -

    <1>    Mode, D, DT, DTS, MT, MTS
    <2>    Initial date to use in internal format
    <3>    Initial time to use in internal format
    <4>    Parent
    <5>    Horizontal alignment against owner (L or R)
    <6>    Value mark delimited array containing the screen size of the owner
    <7>    First year in the year dropdown
    <8>    Last year in the year dropdown

We can override date ranges, set default values and even alter dialog positioning by passing in the correct values.

By Sprezz | Tuesday 11 July 2023 12:17 | 0 Comments

One of the biggest differences between OI TCL and AREV TCL is that the latter made use of the VOC file to allow the user to customise commands and even to add their own commands. This does not seem to be the case in OI, rather a cursory examination of the object code for RTI_IDE_TCL suggests that the command list is hard coded. 

With this in mind it might be useful to document exactly what commands are supported, and where unclear, what they actually do. For the purposes of this article it will be assumed that the reader is familiar with the syntax of AREV TCL commands. When optional variables are nominated they will normally be in the order TABLE ROW(S) OPTIONS.

So in alphabetical order, the commands are as follows - 

CommandDescription
.LDisplay a popup of previously executed commands.
.nCopy the nth command to the command line.
ASSISTANTLaunch the Assistant popup.
ATTACH
ATTACH_TABLE
ATTACHTABLE
Attaches a table.
CLEAR
CLEARSELECT
CLEAR-LIST
Clears the active select list. Optionally if a name is specified, deletes that list.
CLEARFILE
CLEAR_FILE
CLEARTABLE
CLEAR_TABLE
Clears the nominated table.
COPY
COPY_ROW
COPYROW
Copies rows from one place to another. Can use an active select list.
COPYTABLE
COPY_TABLE
Launch the CopyTable dialog ignoring any command line parameters.
COUNT
COUNTROWS
Counts the rows in the nominated table with optional selection criteria
DELETE
DELETE_ROW
DELETEROW
Deletes the nominated rows from the nominated table, or uses an active select list against the nominated table, or launches a dialog.
DELETELIST
DELETE-LIST
Deletes the nominated list.
DICTEither launches the new table dialog or if a table name is specified, opens the dictionary of that table in the IDE.
EDITLaunches the appropriate tool to edit programs or record depending on the nominated table. Will accept a list of keys or an active select list. Particularly useful with FIND (q.v.).
EVALCompiles and runs the statement
FINDFind the nominated string in the nominated table. Produces an active select list.Takes Option (G to produce a grid and (X to produce a list. Useful in conjunction with EDIT.
GETLIST
GET-LIST
Activates the nominated select list.
LISTRuns the LIST statement.
LISTBASIC
BLIST
List the nominated program to screen. Only works with SYSPROCS and requires uppercase row id.
LISTDICTLaunches the LISTDICT dialog prepopulated with any nominated table.
LISTFILES
LISTTABLES
Lists the tables.
LISTINDEX
LIST_INDEX
Lists all indexed tables or the indexes for the nominated table.
LISTVOLUME
LIST_VOLUME
List the contents of the nominated volume.
LISTVOLUMES
LIST_VOLUMES
List all volumes.
MAKEFILE
MAKE_TABLE
Currently not implemented.
MAKEVOC
VOC
Launches the MAKEVOC dialog.
NAME_VOLUME
NAMEMEDIA
NAMEVOL
Names the volume at the nominated location with the optionally nominated name. In the absence of a name a default date time will be used.
OFFCloses OI.
PAINTOpens the form designer for the nominated window.
QUERY_TABLE
QUERYTABLE
Allows selection and activation of a saved list from the query table.
RUNRuns the nominated program passing the nominated parameters.
RUN_REPORTDisplays the run report selection dialog OR actions the passed report name, list command, file info comma delimited.
SAVELIST
SAVE-LIST
Saves the active select list under the nominated name.
SELECT
EXPLODE-ON
Performs the nominated select/explode-on.
SET_MFSLaunches the SET_MFS dialog.
SUMSums the nominated column in the nominated table.
WHODisplays the WHO window.





































By Sprezz | Tuesday 27 June 2023 18:42 | 0 Comments

 Recently we were stumped by a compilation error when using pipes to create a multi line statement, this sort of thing,

    if value then

        on inlist("ALPHA,BRAVO,CHARLIE", value, ",") goSub |

            doAlpha, |

            doBravo, |

            doCharlie

    end

the compiler just kept complaining about an unmatched then/end else. We thought that the only way to get to the bottom of this was to look at what the precompiler produced to see where the error was. Struggling we tried to recall the syntax of the pragma to save the precompiled output. 

Fortunately a more knowledgeable team member pointed out that using the pipe stripped spaces by design, to allow aesthetic alignments of strings. What this meant was that the compiler was not compiling goSub doAlpha, but rather, goSubdoAlpha.

The solution, repeating pipes. Using two pipes prevents the spaces being stripped and 

    if value then

        on inlist("ALPHA,BRAVO,CHARLIE", value, ",") goSub ||

            doAlpha, |

            doBravo, |

            doCharlie

    end

compiled just fine. 

Anyway it's been 16 years since our article on using pragmas and it's still as relevant as ever, so why not check it out and refresh your memory!

By Sprezz | Wednesday 1 March 2023 17:42 | 0 Comments

We recently came across a situation that we have literally never encountered in our decades of working with Linear Hash. Basically a FIXLH operation was interrupted mid process and hi-jinks ensued. The resultant mess took a while to sort out but our findings may prove useful if you ever find yourself in a similar situation.

This specific issue was encountered on an AREV 3.12 system but the logic for FIXLH remains roughly the same so the information remains applicable.

Basically, the FIXLH operation was interrupted about 10% of the way through processing (the NTVDM abended). A quick check of the table header revealed that it believed itself to be a clean empty table with no rows. Yet listing the table produced a set of row ids after an initial pause while nothing was apparently happening.

What had happened was that the table header had been reset to that of an empty table, and the first 10% or so of the primary frames had been blanked. This caused us to investigate the processing undertaken by FIXLH and we can confirm that it is as follows:

    Create and/or clear the temporary dumpfix table

    grab frame zero group

    write rows in group to temporary dumpfix table

    set frame zero header to that of an empty table

    loop

        grab next group

        write rows in group to temporary dumpfix table

        wipe primary frame to null

    until no groups left

    repeat

    copy rows in temporary table back to source table

So if the operation is interrupted, some of the data will be in the source table and some will be in the temporary dump fix table. 

As an additional caveat, note that if the rows in the temporary dump fix table are not stored off elsewhere, they will be lost next time FIXLH is run.

Since the advent of the UD we've never had to actually fix a GFE for quite some time, so this was an eye-opening operation!

Backups are your friend...

Pixel
Pixel Footer R1 C1 Pixel
Pixel
Pixel
Pixel