A word in your shell-like...

Are you sitting comfortably readers? Then buckle yourself in because this is going to be a long ride! 

The Challenge

It all started when Martyn Phillips of Revelation UK approached us for some assistance with toggling button states in his new sample app. Originally he was interested in capturing the various actions that could be used to update the cut/copy/paste buttons in his app and we were able to point him to Carl’s EDITSTATECHANGED blog post, which along with some hints from our own blog on promoting HELP events got him to where he wanted to be.  

Then Martyn asked if we could do something similar for his Save/Clear/Delete buttons. This was a tougher challenge as what we really need is a FORMSTATECHANGED event, not dissimilar to the above, but triggered when the form changes in such a way that the save/clear/delete buttons should be updated. (SPOILER ALERT – it is coming, so treat this blog post as educational not proscriptive!). The problem is, that so many things can change the state of the form. LOSTFOCUS, CLICK, CHANGED, whether it’s a new row, whether it’s an existing row etc, etc. 

So, we concluded that the most viable way would be to promote an event to the level of “all events for all controls” and perform our magic from there.

The Chosen Solution

But this is a sample app, and we’ve already got one promoted event, so we wanted another technique. Musing about how we could possibly know that something had changed the save status of the form, we recalled that the SAVEWARN property is set by the system when the current record is “dirty” and needs to be saved. 

So, we decided to try and intercept the SET_PROPERTY call that updates SAVEWARN. 

Historically we’ve done this using the time-honoured technique of copying $SET_PROPERTY to $RTI_SET_PROPERTY and providing our own shell $SET_PROPERTY. The issue with this, is that updates can overwrite OUR version of $SET_PROPERTY so it’s a maintenance nightmare.

Checking LH logs gave us an idea. The program loader application, RTP27 always seems, when asked to load a program, to look first in the MD file, and if it doesn’t find what it’s looking for, then looks in SYSOBJ. (The reality of the situation is more complex as Revelation’s Bryan Shumsky explained to us when we asked. See below.)

Those of you who are coming to OpenInsight from an AREV or Pick background will know that the MD file (“Master Dictionary”) is simply a collection of pointers telling the system (amongst other things) where to find the program that has just been requested. The traditional format would be as follows:

ROWID

Type of Pointer

Blank

File to find Object in

Name of Object

So, for example the row for SET_PROPERTY would be:

RBASIC

SYSOBJ

SET_PROPERTY.

OpenInsight did away with these pointers and just assumed that executable code would be found in SYSOBJ. But the introduction of the CTO made the MD required again.

Given this information, and the fact that we can modify MD records as we wish, it becomes obvious that we can change the SET_PROPERTY entry to point to OUR replacement for SET_PROPERTY, add a new entry called say, RTI_SET_PROPERTY, and point this at the original SET_PROPERTY. When RTP27 comes to load SET_PROPERTY, it will find SET_PROPERTY (which redirects to ZZ_SET_PROPERTY) which will in turn find RTI_SET_PROPERTY (Which redirects to the original SET_PROPERTY).

So, we would have an MD entry called SET_PROPERTY:

        RBASIC

        SYSOBJ

        ZZ_SET_PROPERTY*PRACTICE_LOG

And an MD entry called RTI_SET_PROPERTY:

        RBASIC

        SYSOBJ

        SET_PROPERTY

So, we can do whatever processing we want (in this case checking amongst other things the SAVEWARN property) and then call RTI_SET_PROPERTY to actually do the real work. (Well in our case, the other way round – let RTI_SET_PROPERTY do its work, then do the checking.)

For an in-depth discussion of this technique, we recommend checking out SRP’s excellent (and amusingly titled) blog post on this subject at https://blog.srpcs.com/hooking-in-basic-using-different-bait/.

We created our MD entries and tested. Logging out and back in to ensure that our new shelled routines were in memory, we were delighted to see that the concept was working. Our routine was being called! 

The First Hurdle

Regretfully, we fell at the first hurdle. Whenever we tried to call our replacement SET_PROPERTY, the system would crash, complaining that we were being passed too many parameters. This seemed ludicrous, I mean, everyone knows that SET_PROPERTY takes 4 parameters, right?

OR so we think – until we take advantage of the wonderful new System Editor’s ability to open Entity OBJECT rows without blowing chunks. ( and dictionary items - thanks SRP!) - let’s give that a try…

Checking the property panel and the function takes FIVE parameters, not FOUR. As you can see, there’s an undocumented “bSetOnly” parameter. Presumably so that we can call SET_PROPERTY in place of SET_PROPERTY_ONLY and have it behave the same, although he who lives by the undocumented feature also dies by it, so make of that what you will. 

We modified our SET_PROPERTY to take five parameters and it was good.

Except it wasn’t. It worked beautifully on single windows, but the moment that you swapped between MDI children or other dialogs, the buttons failed to reflect the current window until data changed. With hindsight this was a rookie mistake. Switching between windows doesn’t generate a SET_PROPERTY, it generates an ACTIVATED event.

So back to square one. 

The NEXT Chosen Solution

In a Windows based environment, all attempts to set a property will come from an event script context, but events also trigger when a window is activated. So, if we could intercept the running of an event, we could not only do our SAVEWARN property tests but we could react to a change of focus between windows, and remove the issue where doing so failed to update the buttons. The routine responsible for this is RUN_EVENT. 

We did some testing and established that RUN_EVENT is called about twice as often as SET_PROPERTY so as there wasn’t an order of magnitude difference we settled upon performing the same shelling trick with as before but with RUN_EVENT. And thereby we entered a world of pain.

A World of Pain, BFSs and Virtual MD Files

The problem with the MD solution described above, is that, unlike in AREV, in OpenInsight the MD file has been made GLOBAL so any changes made in there will affect ALL users of ALL applications. 

We’re great believers in damage limitation, so we thought it’d be much nicer if this redirection only applied when we wanted it to. 

So, we had a choice – we could create an application specific MD and add our pointer rows to that OR we could take advantage of another little tweak highlighted in SRP’s blog post. You see, they documented an area of labelled common utilised by RTP27 to store the file variable of the MD file. It essentially contains three variables, which for the sake of our program we referenced as follows:

Common /RTP27COMMON/ mdFileHandle@, lastFileHandle@, mdHasBeenOpenedAlready@

We knew that we weren’t actually using the MD file anywhere else in our application other than for this very specific redirection, so every time a new program is loaded, we’re wasting a disk read in trying to read the non-existent pointers for the programs that we’re trying to load. So we reasoned that if we could replace the MD file handle with a pointer to our own program, that recognised that it was being called, not to run an event, but to perform as a Base Filing System (BFS) – we could just return the pointer rows ourself and ignore disk reads for all else, thereby providing a small increase in performance overall. 

In theory this was simple, in practice it took a lot of getting right.

The theory behind all file access in OpenInsight (and AREV) is that it is routed via a file handle. The file handle is made up of simply a multi-valued list of MFSs and BFSs, followed by a value mark then any BFS specific data required by the BFS to do its job. 

So, in the case of a normal OpenInsight file with no indexing this would be for example, 

RTP57²000000000062 MP_GPL\DATA\REV75001.LK

for the same file with indexing it would be 

SI.MFS²RTP57²00036000000000093MP_GPL\DATA\REV75001.LK00051DICT.MFS...etc

When an OpenInsight program tries to read using a file handle, the system calls the first program in the file handle (above RTP57 and SI.MFS respectively) passing it the code for the operation required along with the data needed to support the operation. The program then does what it needs to and returns information accordingly. 

In the first instance, our replacement RUN_EVENT code looked at the first parameter being passed, and if it was a number between 1 and 34 it assumed a BFS operation. If the operation was < 3 (so READ or READO) it would check if we were being asked for RUN_EVENT or RTI_RUN_EVENT, and if so return a structure that duplicated the MD entry. In all other cases, this returned a STATUS of FALSE$ indicating that the operation failed. We reasoned that this was safe to do as it would simply tell RTP27 to carry on to read from SYSOBJ.

With our “virtual” MD file in place via calls to our replacement RUN_EVENT, testing could commence once more. And once again it failed. Our new MD pointers were being ignored, because RUN_EVENT was already in memory.

A Little Bit On The Program Loader (RTP27)

To explain the background behind this, when you first want to run a new program, the program loader – RTP27 – is invoked. It looks in the program stack to see if the program is already in memory, and if not if performs the following lookups (information provided by Revelation):

CTO Mode     MD->VOC->SYSOBJ

AREV Mode     VOC->MD->SYSOBJ

OpenInsight Mode MD->SYSOBJ-VOC

When the object code is found it is loaded into the program array. All subsequent calls to load the program will find the program in the program array and use this object code without checking to see if any pointers, or even object code, have changed. 

As we want to modify the RUN_EVENT object code in the array , we need to replace what is already there, with our own modified code. 

The way in which we do this is EITHER to call RTP27 explicitly telling it to reload our two routines (RTI_RUN_EVENT and RUN_EVENT), or to issue a GARBAGECOLLECT which tells OpenInsight to drop everything in memory it doesn’t need and reload when requested. We tried both methods and settled on using RTP27 because using GARBAGECOLLECT on a start-up proc caused our OpenInsight to die!

Armed with this new knowledge we were able to have our version of RUN_EVENT forward to the system RUN_EVENT and then check SAVEWARN and update the buttons accordingly. 

Baby Steps

So, we added an INSTALL method to our RUN_EVENT replacement. It was a very straightforward piece of code:

        modifyMD:

                mdFileHandle@<0, 1> = "ZZ_RUN_EVENT*PRACTICE_LOG" 

                call RTP27(“RTI_RUN_EVENT”)

                call RTP27(“RUN_EVENT”)

                retVal = 1

        Return

Now that the first value in the filehandle is the name of our RUN_EVENT replacement, when I/O operations are performed against the MD file, our program will be called to process it. 

We tested this by writing a small program that included the RTP27COMMON and tried to read rows from the “Virtual” MD file. It all worked as expected so we moved to the next stage. Testing it in real life.

Invoking the System Monitor we installed our virtual MD table (by RUN ZZ_RUN_EVENT “INSTALL”) and we test ran the MP_MDI_MAIN window. To our delight, our program was invoked and behaved as expected. There were some minor glitches related to the actual business logic (see discussion later) but the proof of concept was a success.

Buoyed by this, we sent Martyn a text file containing the program with instructions to compile it and to modify the CREATE event of the MDI form to call our install routine. We sat back and awaited the praise. 

Which was not forthcoming. 

Almost immediately the application started crashing in weird places in RUN_EVENT, referencing unassigned variables which clearly were assigned. It became apparent that something was going badly wrong. Yet it still continued to work fine on the development system.

We were baffled, so we modified our copy of Martyn’s code to do the INSTALL call. Predictably our system now fell over. 

Thus began an infuriating round of logging, testing tweaking which achieved nothing other than frustration. Call the INSTALL from System Monitor, all good. Call from a CREATE event, abject failure. 

Finally, in a developer wide Teams chat, Carl made the (with hindsight obvious) observation, that if the object code for a program was in the returnStack (i.e. it's in use) then RTP27 should not be asked to reload the code. There would be pointers in memory to where the code is currently executing and modifying the object code would invalidate those pointers. 

When you’re in a CREATE event you are obviously inside RUN_EVENT so it shouldn’t be replaced in the array. When you’re in the System Monitor you’re not inside RUN_EVENT so it can be replaced. 

Knowing that the System Monitor can now be treated as an Object in the system, we tried to programmatically run our INSTALL:

$systemMonitor->run( "ZZ_RUN_EVENT INSTALL" )

but this didn’t have the desired effect. The engine was too busy processing our code to respond to the System Monitor. 

Then Aaron had the bright idea to run the install from the system Start-up Proc where it would not yet be in event context, et voilà – the pieces all fell into place – nearly.

Code Coda

The final twist was that something we were doing caused RTP27 to go into a recursive loop and abend OpenInsight. To cut a long story short, we replaced our virtual BFS as an MFS and logged all actions against the MD file. This led to a rewriting of the virtual BFS to return TRUE$ for three specific values and FALSE$ for all others and things finally worked as we expected them to.

Business Logic

So, with everything in place we now had to set about ensuring that our buttons were toggled as required in respect to the state of the current entry window. The buttons we were interested in were the save, the clear and the delete buttons. The save button was easy – is SAVEWARN set? If so, then enable the save button.

The delete button was slightly more complicated. In AREV we would have just looked at WC_OREC% and if it was not empty, then the record could be deleted. (Stating the obvious you can’t delete a new row). Unfortunately, in this release, there is no such property. (Spoiler alert – Rev have added a new WINDOW property called NEWRECORD to the next version.  Set to TRUE$ on the READ event if it's a new record, and reset to FALSE$ on a CLEAR and WRITE). 

The Window common variable origResultRow@ looked as though it had potential and we naively assumed that it would be of the structure RowID : @Fm : Row – and on simple windows we tested, it was indeed. But on more complex windows, sometimes the row id would appear in the middle of an otherwise blank origResultRow@. 

It transpired that origResultRow@ is ordered the same as controlMap@. Therefore, all we had to do to find out whether this was a new row, was to loop through the origResultRow@ variable, grabbing each value and if at the end we had a structure with a field mark it wasn’t a new row!

Penultimately, we wanted to know when the Clear button could be enabled, and after experimentation we landed on using the rowLocks@ variable. If this contains a value then the system thinks it is displaying something (it is not set until you tab off the key prompt).

The final piece of the jigsaw was acting on a close event – this initially didn’t work because all of our code happened after the system CLOSE event, so the window was no longer there to query. This was easily resolved by moving the GET_PROPERTIES we needed to before the RTI_RUN_EVENT.

With all of these things taken into account, our solution was complete.

The code is presented below as an exercise for the student 😊. 

The Final Program

With a reminder as to why you should always use bLen in preference to Len...

2

0001  Function zz_Run_Event(appType,controlID,controlClass,event,arg1,...<more>)
0002      * Copyright (C) 2020 Sprezzatura. All Rights Reserved **
0003  
0004     Declare Function rti_Run_Event, get_Property, retStack
0005     $Insert logical
0006     Common /RTP27COMMON/ mdFileHandle@, lastFileHandle@, mdOpenedAlready@
0007  
0008     if assigned( appType )       else         appType        = ""
0009     if assigned( controlID )     else         controlID      = ""
0010     if assigned( controlClass )  else         controlClass   = ""
0011     if assigned( event )         else         event          = ""
0012     if assigned( arg1 )          else         arg1           = ""
0013     if assigned( arg2 )          else         arg2           = ""
0014     if assigned( arg3 )          else         arg3           = ""
0015     if assigned( arg4 )          else         arg4           = ""
0016     if assigned( arg5 )          else         arg5           = ""
0017     if assigned( arg6 )          else         arg6           = ""
0018  
0019     equ ON$    To 1    ;* Used to enable, or turn on buttons.
0020     equ OFF$   To 0    ;* Used to disable, or turn off buttons.
0021  
0022     * install request/BFS Code, if not process as RUN_EVENT
0023     retVal = TRUE$
0024     stack = retStack()
0025     if stack<2> = "INITIALIZE" then appType = "INSTALL" ; *Startup Proc
0026  
0027     if appType = "INSTALL" then
0028        goSub modifyMD
0029     end else
0030        if num( appType ) and appType # "" then
0031           if appType >= 1 then
0032              if appType <= 34 then
0033                 goSub pretendToBeBFS
0034              end else
0035                 * if the BFS definition is extended to use higher numbers 
0036                 * react appropriately here
0037              end
0038           end else
0039              * if the BFS definition is extended to use lower numbers 
0040              * react appropriately here
0041           end
0042        end else
0043           winID          =         controlID[1, "."]
0044           objxArray2     =         winId
0045           propArray2     =         "SAVEWARN"
0046  
0047           objxArray2     := @Rm :  winID
0048           propArray2     := @Rm :  "MDIFRAME"
0049  
0050           dataArray2      =        Get_Property( objxArray2, propArray2 )
0051  
0052           bSaveWarn      =         dataArray2[ 1, @Rm ]
0053           mdiFrame       =         dataArray2[ bCol2() + 1, @Rm ]
0054  
0055           $Insert OIWIN_COMM_INIT
0056  
0057           lenOldRow      =         bLen( origResultRow@ )
0058           oldRow         =         origResultRow@
0059           rowLocks       =         rowLocks@
0060  
0061           retVal = RTI_Run_Event(appType,controlID,controlClass,...<more>)
0062        end
0063     end
0064  
0065     if event = "TIMER" else
0066        if  controlId[1, 3] = "MP_" and controlId[ 3, 5 ] != "_MDI_" then
0067           * only react to Martyn's windows, except for the MDI Frame
0068           if event _eqc "CLOSE" then
0069               * disable all three buttons, the activated 
0070               * on the next window will set accordingly if there
0071              dataArray2 = OFF$ : @Rm : OFF$ : @Rm : OFF$
0072           end else
0073               * Check whether there was a previous row
0074              ptr = 1
0075              stripped = ""
0076              loop
0077                 nextField = oldRow[ ptr, @Fm, TRUE$ ]
0078              while ptr   <= blen( oldRow )
0079                 /*
0080                    Fun "easter egg" for those who read comments. 
0081                    When evaluating the blen (or the len of an ANSI string) 
0082                    the string length is stored in the descripter so it 
0083                    does not have to be calculated each time. It's arguable 
0084                    that this might be faster than assigning to a variable 
0085                    then using that variable
0086                 */
0087                 ptr       = bcol2() + 1
0088  
0089                 if len( nextField ) then
0090                    stripped := nextField : @Fm
0091                 end
0092              repeat
0093              stripped[ -1, 1 ] = ""
0094  
0095              bExistingRec   =         (index( stripped, @Fm , 1 ) > 0)
0096  
0097              dataArray2     =         bSaveWarn ; * save button
0098  
0099              if len( rowLocks ) then
0100                 dataArray2 := @Rm :  ON$ ; * Clear button
0101              end else
0102                 dataArray2 := @Rm :  OFF$ ; * Clear button
0103              end
0104  
0105              dataArray2      := @Rm :  bExistingRec ; * delete button
0106  
0107           end
0108  
0109           targetWindow      =         if len(mdiFrame) then mdiFrame else winId
0110  
0111           objxArray2        =         targetWindow : ".BTN_SAVE"
0112           propArray2        =         "ENABLED"
0113  
0114           objxArray2        := @Rm :  targetWindow : ".BTN_CLEAR"
0115           propArray2        := @Rm :  "ENABLED"
0116  
0117           objxArray2        := @Rm :  targetWindow : ".BTN_DELETE"
0118           propArray2        := @Rm :  "ENABLED"
0119  
0120           objxArray2        := @Rm :  targetWindow : ".MENU.FILE.SAVE_ROW"
0121           propArray2        := @Rm :  "ENABLED"
0122  
0123           objxArray2        := @Rm :  targetWindow : ".MENU.FILE.CLEAR_FORM"
0124           propArray2        := @Rm :  "ENABLED"
0125  
0126           objxArray2        := @Rm :  targetWindow : ".MENU.FILE.DELETE_ROW"
0127           propArray2        := @Rm :  "ENABLED"
0128  
0129           dataArray2        := @Rm : dataArray2
0130  
0131           call set_Property( objxArray2, propArray2, dataArray2 )
0132        end
0133     end
0134  
0135  return retVal
0136  
0137  modifyMD:
0138  
0139     mdFileHandle@<0, 1>     = "ZZ_RUN_EVENT*PRACTICE_LOG"
0140     call                       rtp27("RTI_RUN_EVENT")
0141     call                       rtp27("RUN_EVENT")
0142  
0143  return
0144  
0145  pretendToBeBFS:
0146  
0147     locate appType In "20,21,22" using "," setting gotIt then
0148        arg3 = TRUE$
0149     end else
0150        arg3 = FALSE$
0151        if appType < 3 then
0152            /*
0153                 READ or READO
0154            */
0155           arg2 = "RBASIC"
0156           arg2 := @Fm
0157           arg2 := @Fm
0158           arg2 := "SYSOBJ"
0159           arg2  := @Fm
0160  
0161           begin case
0162              case event = "RUN_EVENT"
0163                 arg2 := "ZZ_RUN_EVENT*PRACTICE_LOG"
0164                 arg3 = TRUE$
0165              case event = "RTI_RUN_EVENT"
0166                 arg2 := "RUN_EVENT"
0167                 arg3 = TRUE$
0168  
0169           end case
0170        end
0171     end
0172  
0173  return

Post settings Labels No matching suggestions Published on 22/09/2020 16:58 Permalink Location Options