Active X Scripting Host

By Sprezz | Friday, 15 November 2024 14:52 | 0 Comments

An interesting query made its way into the office this week. Our client wanted to be able to validate data using Regular Expressions (REGEXP) for Email validation (^[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\\.[a-zA-Z]{2,}$ if you must know) and so unsurprisingly searched the documentation for REGEXP and came across its usage in RTI_JSON. This led to a series of mutual misunderstandings which led to a far greater understanding of what JSON actually is (see footnote) but still didn't help in actually validating using a REGEXP.

Fortunately, we have resources in the office who were able to advise that what we wanted to achieve was actually relatively simple using the OI ActiveX Scripting Host. This is something that is not yet "officially" documented in the product - but like quite a few things in OI, there is profuse documentation in the Equates. In version 10 this has been handily encapsulated for us the the RTI_AXSH function - the ActiveX Scripting Host. ActiveX Scripting and Active Scripting are the same thing. The X was dropped over the years.

But first - what actually is a Scripting Host?

An Active Scripting Host is a technology used in Windows to support component-based scripting. It allows different scripting engines to be integrated into applications. This enables the execution of scripts written in various languages like VBScript, JScript (Microsoft’s implementation of JavaScript), Python, Perl and others. If we have a task we wish to execute that is better suited to say, JScript - performing a REGEXP for example - we can use RTI_AXSH to do this for us.

To show how easy it is, we'll first provide a small sample program to do this, then explain the program and finally document the routine.

The Code

compile function test_revaxsh_regexp( void )

$insert rti_AXSH_Equates
$insert rti_SSP_Equates
$insert logical

call set_Status( FALSE$ )

errStat = SETSTAT_OK$
errText = ""
hAXSH = 0
retVal = ""

code = 're = new RegExp( "^[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\\.[a-zA-Z]{2,}$" );'
code := 'function testRE( str ) {'
code := ' return re.test( str );'
code := '}'

createParams = ""
createParams<REVAXSH_CREATE_POS_LANGUAGE$> = "JScript"

hAXSH = rti_AXSH( REVAXSH_MTD_CREATE$, createParams )

errStat = get_Status( errText )

if errStat then
return ""
end

call rti_AXSH( REVAXSH_MTD_ADDCODE$, hAXSH, code )

errStat = get_Status( errText )

if errStat then
return ""
end

string = "amcauley@sprezzatura"

if get_Status() else
retVal = rti_AXSH( REVAXSH_MTD_RUNEX$, hAXSH, 'testRE', string )
errStat = get_Status( errText )
end

if ( hAXSH ) then
call rti_AXSH( REVAXSH_MTD_DESTROY$, hAXSH )
end

if errStat then
debug
End

return retVal

The Explanation

The first thing that we need to do is insert RTI_AXSH_Equates. Once this has been done, we construct some simple JScript code in a variable called code. For those not fluent in JScript it's probably worth explaining how the code operates.

We start by create a global variable called "re". Think of it as being like labelled common. It'll be there until the scripting host is destroyed. Loading the code instantiates the re object as a REGEXP type.

We then define a JScript function called testRE so that we can call it from within Basic+.

We initialise our create parameter to tell OI which scripting language to use and then use the CREATE method to initialise our Script Hosting. Now that we have a container to put our code in, we can add it using the ADDCODE method. Assuming no errors we then use the RUNEX method, telling it the name of the function in our code we wish to execute (testRE). We now have the result we want to do with as we will.

Finally in an act of polite housekeeping we tell the system to DESTROY the scripting host we created so as not to leak resources.

We hope that this information makes the Scripting Host a little less scary!

The Documentation - taken from RTI_AXHS_EQUATES.

The RTI_AXSH routine provides a set of methods for interacting with the ActiveX Scripting Host. Below is a detailed guide on how to use these methods, including their parameters, return values, and error handling.

The following methods are supported

Method	Description
CREATE	Creates an ActiveX scripting host instance and returns its handle.
DESTROY	Destroys a script host instance.
GETPROP	Returns the value of a specified property.
SETPROP	Sets the value of a specified property.
ADDCODE	Adds a block of code to the scripting host.
EVAL	Evaluates a statement and returns the result.
EXECUTE	Executes one or more statements (no result is returned).
RUN	Executes a specific script function and returns the result (if any).
GETENGINES	Returns a list of ActiveX script engines installed on the workstation.
ADDOBJECT	Adds a global named object to the scripting host.
RUNEX	Executes a specific script function and returns the result (if any), allowing the script function parameters to be passed as separate arguments.

CREATE

Creates an ActiveX scripting host instance and returns it's handle.

The instance is initialised with the following information:

<1> Script Language (required)
<2> AllowUI (1/0)
<3> hwndSite
<4> Timeout in milliseconds (-1 == no Timeout, otherwise > 0)

e.g.

createParam = "JScript" : @fm : TRUE$ : @fm : @@window->handle
hAXSH = rti_AXSH( REVAXSH_MTD_CREATE$, createParam )

Input	createParam : an @fm delimited dynamic array as specified above
Returns	The handle of the new AXSH instance of successful, otherwise "0".
Errors	All errors are returned via Set_Status()

DESTROY

Destroys a script host instance.

call rti_AXSH( REVAXSH_MTD_DESTROY$, hAXSH )

Input	hAXSH : Handle of the scripting host to destroy [required]
Errors	All errors are returned via Set_Status()

GETPROP

Returns the value of a specified property

e.g.

propName = REVAXSH_PROP_LANGUAGE$
language = rti_AXSH( REVAXSH_MTD_GETPROP$, hAXSH, propName )

Input	hAXSH : Handle of the scripting host to access [required]
Input	propName : Name of the property to access [required]
Returns	The property value
Errors	All errors are returned via Set_Status()

The supported properties are "Language", "Allow UI", "SiteHnd" and "Timeout"

ANY DEFAULTS CARL?

SETPROP

Sets the value of a specified property

e.g.

propName = REVAXSH_PROP_ALLOWUI$

propValue = FALSE$

call rti_AXSH( REVAXSH_MTD_SETPROP$, hAXSH, propName, propValue )

Input	hAXSH : Handle of the scripting host to access [required]
Input	propName : Name of the property to access [required]
Input	propValue : New property value to set.
Errors	All errors are returned via Set_Status()

ADDCODE

Adds a block of code to the scripting host (Any immediate code will be executed as normal)

e.g.

scriptCode = "function add( a, b ) { return a + b; }"

call rti_AXSH( REVAXSH_MTD_ADDCODE$, hAXSH, scriptCode )

Input	hAXSH : Handle of the scripting host to access [required]
Input	scriptCode : Code to add. [required]
Errors	All errors are returned via Set_Status()

EVAL

Evaluates a statement and returns the result.

e.g.

statement = "3+10;"

retVal = rti_AXSH( REVAXSH_MTD_EVAL$, hAXSH, statement )

Input	hAXSH : Handle of the scripting host to access [required]
Input	statement : Code to evaluate. [required]
Returns	The result of the evaluation.
Errors	All errors are returned via Set_Status()

EXECUTE

Executes one or more statements (no result is returned)

e.g.

scriptCode = "function add10( a, b ) { return a + 10; }; add10( 3 );"

call rti_AXSH( REVAXSH_MTD_EXECUTE$, hAXSH, scriptCode )

Input	hAXSH : Handle of the scripting host to access [required]
Input	statement : Code to execute. [required]
Errors	All errors are returned via Set_Status()

RUN

Executes a specific script function and returns the result (if any).

e.g.

scriptCode = "function add( a, b ) { return a + b; }"

call rti_AXSH( REVAXSH_MTD_ADDCODE$, hAXSH, scriptCode )

method = "add"

args = 3 : @rm : 4

retVal = rti_AXSH( REVAXSH_MTD_RUN$, hAXSH, method, args )

Returns	Return value from the function (if any)
Errors	All errors are returned via Set_Status()

GETENGINES

Returns a list of ActiveX script engines installed on the workstation.

e.g.

engineList = rti_AXSH( REVAXSH_MTD_GETENGINES$ )

Returns	An @fm delimited list of installed engines.
Errors	All errors are returned via Set_Status()

ADDOBJECT

Adds a global named object to the scripting host

e.g.

objName = "MyXmlDoc"

objDoc = OleCreateInstance( "Msxml2.DOMDocument" )

call rti_AXSH( REVAXSH_MTD_ADDOBJECT$, hAXSH, objName, objDoc )

Input	hAXSH : Handle of the scripting host to access [required]
Input	objName : "Script name" of the object being added [required]
Input	oleVar : OLE object being added [required]
Errors	All errors are returned via Set_Status()

RUNEX

Executes a specific script function and returns the result (if any), allowing the script function parameters to be passed as separate arguments (This method is the only method that can pass OLE objects to a script function)

The maximum number of parameters that can be passed is 10. Note that this method stops looking for script function arguments when it encounters the first unassigned one.

e.g.

scriptCode = "function add( a, b ) { return a + b; }"
call rti_AXSH( REVAXSH_MTD_ADDCODE$, hAXSH, scriptCode )

method = "add"
arg1 = 3
arg2 = 4
retVal = rti_AXSH( REVAXSH_MTD_RUN$, hAXSH, method, arg1, arg2 )

Input	hAXSH : Handle of the scripting host to access [required]
Input	method : name of the function to execute [required]
Input	arg1 : First parameter to pass to the function
Input	arg2 : Second parameter to pass to the function
Input	arg3 : Third parameter to pass to the function
Input	arg4 : Fourth parameter to pass to the function
Input	arg5 : Fifth parameter to pass to the function
Input	arg6 : Sixth parameter to pass to the function
Input	arg7 : Seventh parameter to pass to the function
Input	arg8 : Eighth parameter to pass to the function
Input	arg9 : Ninth parameter to pass to the function
Input	arg10 : Tenth parameter to pass to the function
Errors	All errors are returned via Set_Status()

A note relating to all scripting errors

The Execute, AddCode, Eval and Run methods can all return parsing and execution errors from the scripting engine. In this case an @svm delimited list of error information is returned with the following structure:

<0,0,1> Error Source
<0,0,2> Error Description
<0,0,3> LineText
<0,0,4> LineNumber
<0,0,5> CharPos
<0,0,6> ContextID

Footnote:

JSON (JavaScript Obect Notation) is a way of serializing JS variables/objects to a human-friendly string format so they can be saved and reloaded. It is not intended as a way to create JS objects and then call their methods and such... despite what the author thought

Pro Tip

CoPilot understands RegExp well. It suggests replacing the REGEXP we started with with "^[\w.%+-]+@[\w.-]+\\.\w{2,}$".

Feeling hot hot hot...

By Sprezz | Wednesday, 3 July 2024 10:09 | 1 Comments

Recent queries on the Works forum led us to realise that there isn’t one single document that describes the various ways (well two) that you can interact with OLE controls within OI10. So we thought that we’d put it all together in one place using as an example a third party control called CSxGraph which you can download here . This is actually an ActiveX control, rather than an OLE control but that’s primarily a Microsoft branding exercise from nearly 30 years ago now, 1996! So, if you see something described as an ActiveX control, just assume it’s an OLE control.

Put simply, OLE components can be added to forms and manipulated via the standard Presentation Server functions, GET and SET_PROPERTY, EXEC_METHOD etc, or they can be created programmatically and be manipulated using intrinsic R/Basic routines such as OLEGETPROPERTY. If you want the user to see the control and possibly interact with it, you’ll need to add it to a form. If that isn’t a requirement then you’d instantiate the control programmatically.

OLE on Forms

To add an OLE control to a form, simply select the OLE Control menu item, and position the control where you want it on the form.

As this is a new OLE control, the system needs to know which OLE control to use, so it presents a dialog from which you can choose the control to insert

If you know the CLSID or the ProgId of the control you can enter it here, or you can use the options button to select from a dialog, cleverly only showing the OLE objects that can be added as controls. In this case we know that we’re working with csXGraph to we can select that from the dialog

OK this, then OK the initial dialog, the system inserts the OLE control as requested

There is an options button next to the CLSID which enumerates all the properties, methods and events that the control exposes, along with additional general information about the control type

So, it’s possible to work with the control without checking the documentation for a lot of things. Whilst this dialog provides a handy summary of what’s available, it’s more like a table of contents, pointing to additional items in the property panel. Further down the property panel we find OLE Properties.

Interacting with the UI

All of the OLE Properties that are exposed by the control can be found in the OLE Properties section of the property panel. We can set these properties manually in the property panel at design time or we can alter these properties at run time programmatically.

The property list is comprehensive as you can see

But, if we know that we want this to be a Pie Chart we could select the graph type and choose that option

Similarly, we could set the title to a known value

And if we run the form, having set these properties,

Nothing happens…

This is to be expected, we’ve told the control what title we want and that we want it to draw a pie chart, but we haven’t given it any data to display (properties), and more importantly, we haven’t told it that we actually want it to display anything (methods).

So let’s add an edittable to capture our pie slice values and add a button to actually grab that data and display it.

To achieve this, we need to firstly get the data which we wish to display, then grab the title and amend it to include the updated date and time, clear the existing data (if any) from the OLE Control and then for each line of data to display, update the data in the pie chart. Finally, we need to tell the control to plot and display the results.

In the interests of transparency, the device this is being written on has a High DPI screen and this control is not High DPI aware so some manual playing with properties in the panel was required to make it visually – I won’t say appealing – appropriate.

Getting the data

This is a normal OI GET_PROPERTY call

Changing the title

This still uses normal OI GET and SET properties, but to ensure there is no clash between a PS property and an OLE property, we explicitly tell the PS that we’re wanting an OLE property by prepending the property name with “OLE.”

Clearing the data

For this we use EXEC_METHOD function, but as with the GET and SET we qualify our method name

Updating the data

Again we’re using EXEC_METHOD and providing random colours for the slices

Plot and display

And finally again an EXEC_METHOD

Interacting with Events

That’s the UI element dealt with but of course we might want to respond to an event on the control. For example, clicking the mouse down button on the graph, or maybe doubleclicking on an area of the graph.

For this we have to turn to the property panel again, but this time the OLE Integration property

As you will see, by default there are no events qualified, so when the events ARE triggered, they will not raise an OLEEVENT on the control.

If we were interested in trapping the mouse down event and the doubleclick event, we would simply launch the QualifiedOLEEvents dialog and check those events.

Now when the control is double clicked, or the mouse is clicked down on the control, an OLEEVENT will be raised and the parameters documented will be passed to your handling routine, be it a script or a commuter.

So if we put a debug in the OLEEVENT script for a control and mouse down over the control we will see this

With the EventName being onMouseDown, and with the parameters being (according to the control documentation)

Param1 Mouse Button
Param2 Shift state
Param3 Pixel X
Param4 Pixel Y
Param5 Graph X
Param6 Graph Y
Param7 Pixel Colour

Moving on to other elements of the event setup - the three Sync Types are Synchronous, Asynchronous and Callback.

By default, the sync type is set to Asynchronous. An asynchronous event is queued and is thus guaranteed to be executed once everything before it in the queue has been dealt with. This can lead to a slight lag if the application is busy.

Setting the event to Synchronous will attempt to execute it immediately, but if the engine is tied up doing anything else it will just fail.

Setting the event to Callback creates a sort of hybrid situation where the system will attempt to execute the event immediately, but if it fails it will establish if the engine is waiting on a response from the Presentation Server rather than the engine being tied up running a program. If it is waiting, it will branch off and run the event there and then.

So synchronous can be used when failure is immaterial, asynchronous can be used when failure is not an option and callback can be used when failure is optional but speed is of the essence.

For a discussion of the other check boxes see OLE event [Revelation Wiki], but essentially –

SuppressName	ShowDispID	Result
False	False	Returns the event name and any arguments
True	True	Returns the DispID (the numeric constant representing the event type) and any arguments. Using a numeric constant makes branching faster, as it removes the requirement for string comparisons.
False	True	Returns the event name, the DispId and the arguments. Permits mapping between the DispId and the name.
True	False	Of limited use as it returns nothing.

OLE in Programs

Programmatic OLE control allows us to automate our applications, driving external programs to produce results that would be hard to achieve in pure OI. Sending Emails, performing mail merges, generating images etc.

Using OLE in programs is normally just a case of “instantiating” the OLE control – creating and initialising it – setting relevant properties and telling it to run the required methods. It is less likely that we would need to respond to OLE events in this context.

To illustrate programmatic usage, we will again return to our csXGraph control, but this time we will programmatically generate the graph and save the image.

To manipulate OLE controls programmatically we make use of the following self-evidently named routines –

OLECREATEINSTANCE    - to instantiate the OLE control
OLEPUTPROPERTY        - to set a property of the control
OLEGETPROPERTY        - to get a property of the control
OLECALLMETHOD        - to call a method of the control
OLESTATUS                - to query the status of an OLE operation.

Note that these routines are intrinsic to the compiler and so do not need to be declared. Note further that these routines must be called individually for each property or method. It is not possible to stack the parameters in an array using @Rms and make a single call as is possible with the relevant PS routines.

OLESTATUS

This function takes no parameters and returns the status of the last OLE operation. If the value if FALSE$ then the operation succeeded otherwise a negative integer representing the error code is returned.

This error code is not particularly helpful, so it helps to use the system function RTI_ErrorText to retrieve an English language description. This function takes three parameters. The kind of error being retrieved, the error code in question, and whether Carriage Return/LineFeed should be swapped out for spaces.

Status = oleStatus()
If status then errorText = RTI_ErrorText(“WIN”, status, FALSE$)

It should be used after every OLE operation.

OLECREATEINSTANCE

This function takes a single parameter – being the name of the class of the OLE control that we wish to instantiate.

So for example

oleObj = oleCreateInstance( “csXGraphTrial.Draw” )

If there was no error then we now have an OLE control instantiated in memory to play with. The oleObj variable will contain the handle for the OLE control.

OLEPUTPROPERTY

This routine takes a polymorphic set of parameters, depending upon whether the OLE object is indexed. If it isn’t, then it takes the standard three SET_PROPERTY parameters, namely the OLE object handle, the property and the value to set.

If the property is indexed (and it’s important to realise that this might not just be X and Y indexes – the object could be n dimensional), then the index values follow the property name, so

olePutProperty( oleObj, property[, index1, index2,…], value)

OLEGETPROPERTY

This routine takes a polymorphic set of parameters, depending upon whether the OLE object is indexed. If it isn’t, then it takes the standard two GET_PROPERTY parameters, namely the OLE object handle, and the property to get.

If the property is indexed (and it’s important to realise that this might not just be X and Y indexes – the object could be n dimensional), then the index values follow the property name, so

Return = oleGetProperty( oleObj, property[, index1, index2,…])

OLECALLMETHOD

This routine takes two required parameters – the OLE object handle and the name of the method to call. If the method requires parameters then these are passed as optional parameters to the OLECALLMETHOD call.

OleCallMethod( oleObj, methodName[, param1, param2…])

Bringing it all together

For our example, it wasn’t apparent from the screen shots, but several default properties had been changed to make the graph more attractive. We need to replicate this in our code. The properties that were changed were primarily to do with the size of the image, the title and obviously the graph type. It doesn’t really matter what order we set the properties in as long as all required properties are set before we ask the control to execute any methods.

So alphabetical is as good as any.

CenterX
CenterY
GraphType
*Height
LegendX
LegendY
MaxX
MaxY
Offset
OriginX
OriginY
PieDia
ShowLabel
ShowLegend
Square
Title
TitleX
TitleY
Width

The “*” is used to draw attention to the fact that although this property is named identically to the form designer property, there will be no confusion as we are setting and getting OLE properties directly without going through the PS.

So now we need to modify the program to set the required properties.

Note that there’s a debug in the checkOLEStatus. Check the code above to see what line you think it’ll break at when the program is run. Well, the line with the debug in checkOLEStatus obviously, but who was the culprit?

So we run the code, there is an error, and obviously it’s -2147352570 error….

Which using the aforementioned RTI_ErrorText tells us is an “Unknown name”… note that the final version of this program uses OLE in place of WIN as the error type – the two are interchangeable but using OLE makes more sense in an OLE context.

And sure enough “PiaDia” is an error – it should be PieDia.

We can rectify this then start to add the data.

To add the data, we first tell the control to use randomised colours – so we can see that the graph is a different screen shot than the Form Designer one. We then use oleCallMEthod to add our four data values.

Then we tell the OLE control to draw the graph

And finally, we tell it to save the graph to a BMP file

The end result?

Hopefully armed with the foregoing you'll now be adequately equipped to tackle your own OLE projects, but if you want to play the code for this article follows.

The Demo Program

Subroutine zz_OLE_Demo( Void )

Declare Function RTI_ErrorText
$Insert logical

oleObj = Olecreateinstance( "cSxGraphTrial.Draw") |
; Gosub checkOLEStatus

If ok Then olePutProperty( oleObj, "CenterX", 350 ) |
              ; Gosub checkOLEStatus
If ok Then olePutProperty( oleObj, "CenterY", 350 ) |
              ; Gosub checkOLEStatus
If ok Then olePutProperty( oleObj, "GraphType", 0 ) |
              ; Gosub checkOLEStatus
If ok Then olePutProperty( oleObj, "Height", 700 ) |
              ; Gosub checkOLEStatus
If ok Then olePutProperty( oleObj, "LegendX", 100 ) |
              ; Gosub checkOLEStatus
If ok Then olePutProperty( oleObj, "LegendY", 50 ) |
              ; Gosub checkOLEStatus
If ok Then olePutProperty( oleObj, "MaxX", 500 ) |
              ; Gosub checkOLEStatus
If ok Then olePutProperty( oleObj, "MaxY", 500 ) |
              ; Gosub checkOLEStatus
If ok Then olePutProperty( oleObj, "Offset", 20 ) |
             ; Gosub checkOLEStatus
If ok Then olePutProperty( oleObj, "OriginX", 200 ) |
              ; Gosub checkOLEStatus
If ok Then olePutProperty( oleObj, "OriginY", 200 ) |
              ; Gosub checkOLEStatus
If ok Then olePutProperty( oleObj, "PieDia", 400 ) |
              ; Gosub checkOLEStatus
If ok Then olePutProperty( oleObj, "ShowLabel", False$ ) |
              ; Gosub checkOLEStatus
If ok Then olePutProperty( oleObj, "ShowLegend", True$ ) |
              ; Gosub checkOLEStatus
If ok Then olePutProperty( oleObj, "Square", 16) |
              ; Gosub checkOLEStatus
If ok Then olePutProperty( oleObj, "Title", "OI 10 OLE Demo" ) |
              ; Gosub checkOLEStatus
If ok Then olePutProperty( oleObj, "TitleX", 20 ) |
              ; Gosub checkOLEStatus
If ok Then olePutProperty( oleObj, "TitleY", 20 ) |
                  ; Gosub checkOLEStatus
If ok Then olePutProperty( oleObj, "Width", 700 ) |
              ; Gosub checkOLEStatus

// now add the data

If ok Then olePutProperty( oleObj, "UseRNDColor", TRUE$ ) |
; Gosub checkOLEStatus
If ok Then olePutProperty( oleObj, "RNDColor", 25 ) |
; Gosub checkOLEStatus

If ok Then retVal = oleCallMethod( oleObj, 'AddData', "Cherries", 48, 0) |
              ; Gosub checkOLEStatus
If ok Then retVal = oleCallMethod( oleObj, 'AddData', "Apples", 16, 0) |
              ; Gosub checkOLEStatus
If ok Then retVal = oleCallMethod( oleObj, 'AddData', "Peppers", 4, 0) |
              ; Gosub checkOLEStatus
If ok Then retVal = oleCallMethod( oleObj, 'AddData', "Aubergines", 2, 0) |
              ; Gosub checkOLEStatus

// and tell it to draw the graph

If ok Then retVal = oleCallMethod( oleObj, 'DrawGraph' ) |
              ; Gosub checkOLEStatus

// and save to file

If ok Then retVal = oleCallMethod( oleObj, 'SaveToFile', "ZZ_OLE_DEMO.BMP" ) |
; Gosub checkOLEStatus

Return

checkOLEStatus:

OK = TRUE$
Status = Olestatus()
If Status Then
debug
errorText = RTI_ErrorText("OLE", Status, FALSE$ )
OK = FALSE$
End

return

A Forest Of Logs

By Sprezz | Monday, 4 March 2024 09:27 | 0 Comments

Since the sad demise of Revelation's article on the subject, there seems to be no quick reference guide available for the various kinds of log that can be created by OpenInsight to assist in debugging strange occurrences and things that go bump in the night.

As a service to the Rev community we therefore offer up our guide to all things log related. Speaking of logs, my best friend at school was nicknamed "Beaver". Steven Norwood. Oh the fun we had in maths lessons (this was pre-calculator days so we had to use log books)... Anyway

Event Logs

These logs are initiated when the "Start Log" button is pressed on the engine. The log file is created under the OpenInsight subdirectory.

Version 9

The file is named OELOGNNN.LOG and will roll over at 1000.

The log will contain every event triggered and all programs run not called from BASIC+ along with the passed parameters. For example

Start OpenEngine log - 2/24/2024 13:55:03

******************************************************************************

Begin processing time: 2/24/2024 13:55:10

RUN REPOSITORY 'WRITE', #1, #2, #3, #4, #5, #6, #7, #8, #9, #10, #11

#1: 'SYSPROG*OIWIN**ZZ_TEST'

#2: '0'

#3: ''

#4: ''

#5: ''

#6: ''

#7: ''

#8: ''

#9: ''

#10: 'ZZ_TEST'

#11: '300þ1þÿZZ_TESTýýWINDOWýý574ý258ý-484ý-311ýUntitled...'

Stop processing time: 2/24/2024 13:55:10

Execution time: 250 milliseconds.

Execution status: successful.

******************************************************************************

Begin processing time: 2/24/2024 13:55:10

RUN RUN_EVENT 'SYSPROG*OIWIN', 'ZZ_TEST.EDITLINE_1', 'EDITFIELD', 'GOTFOCUS*3*SYSPROG*GOTFOCUS..OIWIN*', 'ZZ_TEST.EDITLINE_1'

Stop processing time: 2/24/2024 13:55:10

Execution time: 46 milliseconds.

Execution status: successful.

******************************************************************************

The program's parameters are listed individually whereas the parameters for the event are passed in line. It should be noted that the engine log only logs programs that are not called directly from Basic+ so it is not as useful as it might seem for debugging.

Version 10

The file is named REVENGINENNN.LOG and will roll over at 1000.

The log will contain every event triggered with the passed parameters. For example

Start RevEngine A log - 2/24/2024 14:09:40

******************************************************************************

Begin processing time: 2/24/2024 14:09:45

RUN RUN_EVENT 'SYSPROG*OIWIN','RTI_IDE','WINDOW','ACTIVATED*2*SYSPROG*ACTIVATED.WINDOW.OIWIN*'

Stop processing time: 2/24/2024 14:09:45

Execution time: 16 milliseconds.

Execution status: successful.

******************************************************************************

Due to the fact that OpenInsight is now written in OpenInsight, all program calls originate from Basic+ so we no longer see programs being launched. Previously the various tools were external executables that relied upon calls to the engine to execute Basic+ code. These calls could be tracked in the log. Now that the tools are written in OI they don't make calls to the engine to execute code, they just call the code directly.

Profile Logs

As the name suggests, these logs record every single program called and returned to, along with granular timings, whilst the logs are active. In OI 9 these logs can only be recorded for the entirety of the OI session. In OI 10 it is possible to toggle them. In both versions the creation of a dummy text file in the OpenInsight subdirectory is required to record the log for the session.

Once you have this log it is possible to analyse the information to zoom in on where you may be experiencing speed issues, recursive code et al.

NB These logs slow the program speed down exponentially so it is a good idea to remove the dummy text file when it is no longer used (for example after you have started logging).

Version 9

To trigger the recording of a log simply create a text file called OEPROFILE.LOG before logging into OpenInsight. The system will create a file called OEPROFILE_MACHINENAME_PROCESSNUMBER.LOG. All timings in the log are in milliseconds - but only to a granularity of 16ms - hence the zeroes in the log.

As an example of a log starting OpenInsight here's the first few lines -

LOAD_SYSPROGDBT 0

RTP57 16

GETNETWORKTYPE 16

GETNETWORKTYPE 31 15

RTP57A 31

RTP57A 47 16

RTP57A 47

RTP57A 47 0

RTP57A 47

RTP57A 47 0

RTP57 47 31

RTP50 47

RTP50 47 0

What this tells us is that the program LOAD_SYSPROGDBT has started, and the first thing it did at tick 16, was to call RTP57, which in turn called GETNETWORKTYPE which took 15 ticks to complete. Some quick calls follow - some too fast to measure, until RTP57 exits 31 ticks after it started.

For reference here is another section where RUN_EVENT is referenced.

RUN_EVENT 2044

RTP27 2044

GETOISTATE 2044

GETOISTATE 2044 0

RTP57 2044

RTP57A 2044

RTP57A 2044 0

RTP57 2044 0

RTP27 2044 0

This will become relevant in the Version 10 section.

Whilst the log is useful, there are no tools provided for consuming it.

Version 10

To trigger the recording of a log simply create a text file called REVPROFILE.LOG before logging into OpenInsight. The system will create a file called REVPROFILE_MACHINENAME_PROCESSNUMBER.LOG.

As an example of a log starting OpenInsight here's the first few lines -

LOAD_SYSPROGDBT 27.36

RTP57 32.16

GETNETWORKTYPE 32.23

GETNETWORKTYPE 33.95 1.72

MSWIN_GETUSERNAME 33.99

MSWIN_GETUSERNAME 34.79 0.79

MSWIN_GETCURRENTPROCESSID 34.83

MSWIN_GETCURRENTPROCESSID 34.85 0.02

RTP57A 34.88

RTP57A 35.58 0.71

RTP57A 35.61

RTP57A 35.63 0.01

RTP57A 35.64

RTP57A 35.66 0.02

RTP57 35.69 3.53

RTP50 35.71

RTP50 35.73 0.02

The first thing that jumps out is that the timings are now to two decimals places, and they're no longer in ticks, rather they're in 0.01ms. So 1,600 times more granular.

The second thing that jumps out is when looking at a RUN_EVENT call -

RUN_EVENT 5153.86 ("SYSTEM", "LOGIN", "1", "SYSPROG", "SYSPROG", "SYSPROG")

RTP27 5153.87

GETOISTATE 5153.88

GETOISTATE 5153.88 0.00

We are now actually provided with what RUN_EVENT was doing.

Version 10 also has an analysis tool for the log files AND provides for manipulating the logging programmatically. Rather than reinvent the wheel we just refer you to the source.

Linear Hash Logs

These logs provide a record of every single file i/o operation performed at the workstation doing the logging. As with profile logging, linear hash logging is initiated by the creation of a specifically named file at the OpenInsight location. Unlike with profile logging, linear hash logging at the client is performed only for the first person to log in after the creation of the trigger table, so there is no performance impact on other users. If you wish to record all file i/o operations for all service users then server logs should be used instead qv but be warned these are very verbose so get very big very quickly.

UD3

Client

The trigger file in this case is LH3.LOG.

As an example of an LH3.LOG file, here's the first few lines

SN REVMEDIA OK SU OK UL 0 OK OP REPOSIX 1 OK OP REVDICT 2 OK OP REVREPOS 3 OK LO 3 U47961 OK UL 3 U47961 OK LO 3 U47961 OK SU OK OP REVMEDIA 0 OK OP DATAVOL\REVMEDIA 4 OK OP AREV_DIR\REVMEDIA 5 OK OP O4WFILES\REVMEDIA 6 OK RO 0 SYSOBJ*GLOBAL OK OP REV30000 7 OK

Each line represents a single i/o operation, with the first value being an opcode that determines the subsequent parameters.

Looking at some of the obvious operations, OP REVMEDIA 0 OK, means open the REVMEDIA file to handle 0. Then RO 0 SYSOBJ*GLOBAL OK, means make a cached read (READO) of the file entry for SYSOBJ*GLOBAL from file handle 0 - the REVMEDIA map.

Regretfully there are no system tools to analyse the log files.

Server

The trigger file in this case is LH.LOG.

As an example of an LH.LOG, here's an exercise for the reader....

UD4

Client

The trigger file in this case is LH4.LOG.

As an example of an LH4.LOG file, here's the first few lines

SN REVMEDIA OK SU OK UL 0 OK OP REPOSIX 1 OK OP REVDICT 2 OK OP REVREPOS 3 OK LO 3 U47961 OK UL 3 U47961 OK LO 3 U47961 OK SU OK OP REVMEDIA 0 OK OP DATAVOL\REVMEDIA 4 OK OP AREV_DIR\REVMEDIA 5 OK OP O4WFILES\REVMEDIA 6 OK RO 0 SYSOBJ*GLOBAL OK OP REV30000 7 OK

Notice the subtle difference from the UD3 log? That's right, there isn't one.

Server

The trigger file in this case is LH47SRVC.LOG.

As an example of an LH47SRVC.LOG, here's the first few lines, in a smaller font to prevent wrapping - it is verbose

Initialize...done.

Start pending...Running.

Connect...client 0.

0:Version Check...done.

0SRVC Connecting...passed parameter X99999999|0

Validating...bValid 1, bAuthorized 1

:Open Session,X99999999,250...session 0.

0,0,ZZ\ZZ15\REVMEDIA:Open File (resolved to E:\ZZ\ZZ15\REVMEDIAðNf)...file 0.

0,0:Unlock All...done.

There are some gotchas analysing these files as the first two lines are CRLF delimited and all subsequent are LF delimited, but the verbosity of the text makes understanding what is going on a lot easier.

UD5

Client

The trigger file in this case is REVLH.LOG.

As an example of a REVLH.LOG file, here's the first few lines

REVMEDIASU OK

UL 0 OK

OP REPOSIX 1 OK

OP REVDICT 2 OK

OP REVREPOS 3 OK

LO 3 U50707 OK

SU OK

OP AREV_DIR\REVMEDIA 4 OK

OP LH_DIST\REVMEDIA 5 OK

OP O4WFILES\REVMEDIA 6 OK

OP REVMEDIA 0 OK

OP DATAVOL\REVMEDIA 7 OK

OP FAQS\REVMEDIA 8 OK

RO 0 SYSOBJ*GLOBAL OK

This time there IS a subtle difference - there's a weird bug where it puts the CRLF after the SN and swallows the OK., Presumably an offset issue in the osbwrite. Also the login semaphore logic is slightly different. Beyond that the log file follows the pattern of the others.

Random trivia - logging in and out of 9 generates a 10Kb log file. Doing the same on 10 generates a 168Kb log file. Now that OI is written in OI there's a lot more i/o needed to bootstrap the system.

Useful random trivia - you can restrict the operations recorded by creating a file called REVLH.DETAILS containing a comma delimited list of the opcodes you wish to capture. The log will then only contain those opcodes and be correspondingly quicker to generate.

Server

The trigger file in this case is REVLHSRVC.LOG.

As an example of a REVLHSRVC.LOG, here's the first few lines, with wrapping left in place.

Initialize...done.

Start pending...

VSS: UDVSSWriter Initializing...

Running.

[156699562] initialized

[156699562] [156711625] Connect... [156711625] client 0.

[156711625] [156711625] 0 [156711625] :Version Check... [156711625] OK; secure channel check (0\0)... [156711625] done.

[156711625] [156711625] 0 [156711625] NUL Connecting...passed parameter W99999999|1

[156711625] [156711625] Validating...bValid 1, bAuthorized 1

[156711625] [156711625] :Open Session [156711625] , [156711625] W [156711625] 9 [156711625] 9 [156711625] 9 [156711625] 9 [156711625] 9 [156711625] 9 [156711625] 9 [156711625] 9 [156711625] ,9 [156711625] ... [156711625] session 0.

[156711625] [156711625] 0 [156711625] ,0 [156711625] ,E:\OpenInsight\UD Test\REVMEDIA [156711625] :Open File (resolved to E:\OpenInsight\UD Test\REVMEDIA)... [156711640] file 0.

[156711640] [156711640] 0 [156711640] ,0 [156711640] :Unlock All... [156711640] done.

[156711640] [156711640] 0 [156711640] ,0 [156711640] ,E:\OpenInsight\UD Test\REPOSIX [156711640] :Open File (resolved to E:\OpenInsight\UD Test\REPOSIX)... [156711640] file 1.

[156711640] [156711640] 0 [156711640] ,0 [156711640] ,E:\OpenInsight\UD Test\REVDICT [156711640] :Open File (resolved to E:\OpenInsight\UD Test\REVDICT)... [156711640] file 2.

[156711640] [156711640] 0 [156711640] ,0 [156711640] ,E:\OpenInsight\UD Test\REVREPOS [156711640] :Open File (resolved to E:\OpenInsight\UD Test\REVREPOS)... [156711640] file 3.

[156711640] [156711640] 0 [156711640] ,0 [156711640] ,E:\OpenInsight\UD Test\REVMEDIA [156711640] :Open File (resolved to E:\OpenInsight\UD Test\REVMEDIA)... [156711640] file 0.

There are some gotchas analysing these files as all lines are CRLF delimited apart from the VSS line and the Running line which are CR delimited, but the verbosity of the text makes understanding what is going on a lot easier.

Additional Tools

A big shout out to Revelation who made us aware of an additional tool allowing you to observe LH Statistics directly from the workstation. Not only that - you can dynamically turn linear hash logging on or off from the tool.

To use EXEC RTI_LH_INFO_VIEWER at TCL and the following window displays -

There is lots of information available to you here so explore at leisure! To start logging simply select "Client" or "Server" from the "Logger" dropdown and then press "Go". To stop logging, select "Stop Logging" from the "Action" dropdown and then press "Go".

Finally - log files can get a bit overwhelming so if you want to set a "place marker" in a linear hash log, just read a non-existent row using an easily recognisable dummy row id like "ZZ_STARTS_HERE" then you can search the log for that without wading through lots of extraneous stuff.

NosOpen

By Sprezz | Wednesday, 7 February 2024 16:32 | 0 Comments

Recently we were playing with a routine to recurse through directory structures on a nominated disk and attempt to osOpen a specific file at a location - a quick utility to hunt down rogue OpenInsight installations by looking for SYSPROG DBTs. What started off as a vanity project very quickly became a learning exercise about how the system handles OS files behind the scenes.

The routine was working fine, correctly identifying locations containing OpenInsight but then we tried pointing it at Drive C. Suddenly the routine was finding rogue OpenInsight installations all over the disk, SYSPROG.DBTs were opening like they were a dime a dozen. Rogue installation-a-rama.

Except they weren't actually there. OsOpen claimed they were there - well it reported that it had successfully opened the file - but on closer inspection there was no file actually there.

There seemed to be no great mystery about what the apparent rogue installations had in common - they all had humongously long folder paths. Once the path gets beyond a certain length, attempts to open files that aren't there succeed. Rather than keep you in suspense we'll just confirm that this length is a common Windows constant called MAX_PATH - which if you check your msWin_MAX_PATH_Equates insert (yes - there is one) equates to 260 characters.

We reported this to those good folks at Revelation and further inspection revealed that, yes, there is an issue when opening files whose locations are longer than 260 characters. It was explained that internally the system stores the OS handle in an array of up to 128 values and that these values were predeclared at MAX_PATH plus a bit.

This interested us, because we never realised that there was a limit of 128 concurrently open OS files in the Rev product line. We'd always assumed that (given that the osOpen file handle is just the name of the file) the system opened and closed the file each time it was written to or read from. That assumption was incorrect as, as explained above, Rev keep an array of OS file handles. They take the osOpen handle, look it up in an array and extract the OS file handle.

This got us thinking, because we aren't always diligent about osClosing handles. Why had we never seen an issue with over 128 files? So we set about testing our assumptions.

It transpires that, if the system uses up the 128 handles, it starts to reuse the array slots in a FIFO manner. So if you subsequently try an OS operation on a handle that has been dropped, Rev reopens it and drops another.

The implication of this (which we tested to be true) is that osOpen is not actually needed for OS operations.

We don't recommend NOT using it we hasten to add, this was just an academic exercise which could be invalidated at any time.

As an aside when Rev gets the OS handle it places a lock on the file in question. You can see this if you attempt to access the file from outside of OpenInsight. You'll see a message like this -

The lock is removed when either you osClose the file or the system does so during its FIFO operation. A good tip from Rev is that, and I quote

"when logging to a file I like to use

osOpen myFile to myHandle

offset = dir(myfile)<1>

oswrite myChunk on myHandle at offset

osClose myHandle

This avoids holding the lock, so you can use notepad++ to look at the changes while a process is running.".

Anyway you'll be pleased to know that for subsequent releases of OpenInsight the restriction on long path names has been removed - along with the 128 element array limitation.