One of our current mega-projects is converting a very large AREV system over to OpenInsight and of course like most large systems it relies heavily on batch processes that perform selects and refine these selects before presenting the results. AREV had the useful status line update facility to let you know how processing was going but regretfully OpenInsight omitted this - presumably because lacking a dedicated status line there was nowhere to put it. With hindsight a SEND_INFO every 1% would at least have allowed the developer to intercept and update the information from a select.

We tried the usual approach of looking the other way but when some of the queries were taking upwards of five minutes to return results to the user we decided that enough was enough.

The problem as we saw it was split into two areas - selects against entire non-indexed files and selects against indexes. This article deals with the former. The next article in this series will deal with the latter.

To start let's be frank. This blog isn't about giving away the family silver. It's about telling you what is possible using Rev products and giving you enough information to do it yourself if you want to. To this end we won't be publishing reams of code - we'll be publishing snippets and explanations of how to achieve the end result!

The steps involved
At its simplest the easiest way of providing the user with a progress indicator when reading through a file or a resolved active select list is as follows :- include within the select statement an additional column which makes a callback to a user defined routine to provide feedback to the user about where we are in the process. Like most consultancy houses, Sprezz has a generic progress window that we use to keep the user informed of progress through a process. This is called in one of four ways :-

• To start the progress window
• To update the progress window
• To update the log portion of the progress window
• To end the progress window

Thinking about this, this means that for our select statement to report back to us, we need to do the following :-

• Initialise our counter variables and any labelled common
• Start our progress window
• Call our rlist select command
• From within the rlist select command call back to a routine to update the progress window
• On termination close down the progress window

Now that's all very well, but it's a faff to have to go in and change all of our existing calls to RLIST to do this set of actions, so it makes sense to create our own select program that encapsulates all of the above steps.

The Result
Having done this we can then just issue a call to our replacement routine passing it the select statement required, and Bob as they say, is the brother of your mother. So in our simple example we issue the command :-

Call gen_Select_With_Progress( "SELECT CLAIMS" )

and the result we see is :-


(Please excuse the background colour of the MDI Frame. Like most development shops we use three systems, development, testing and live and we colour the background vividly to remind the person working on the system just where they currently are!)

The Breakdown
So looking at the important parts, let's see how they work - firstly let's look at a simplified version of our gen_select_with_progress. Our full version has more flexibility built in but we've stripped this down to the bare bones to make it easier to understand.

The shell program


That copes with list items 1, 2, 3 and 5 so how does the CALLBACK dictionary item work?

CALLBACK Dictionary Item
If you place a compiled dictionary item into the SYSDICT table it can be used in all of your SELECT statements. All that this has to do is to call your update program. Again a simplified version of our SYSDICT item...



As an aside, for efficiency we'd be better off using the transfer statement to move @RECORD and @DICT in the above code as all strings over 20 characters in length are held on the heap rather than in a descriptor, so a copy statement wastes resource.
The Update Routine
Finally the part of gen_select_with_progress that updates the progress meter.



The only thing to mention is that loopCounter@ is a labelled common variable. Because the update program and the calling dictionary item are on the program stack there is minimal overhead to this approach. It also allows for the user to press a cancel button and exit a protracted select. In our testing we've seen this add anywhere between 5 and 20% overhead. This might seem like a lot but when a user is staring at a blank screen waiting for a result time drags a lot more slowly. In addition the estimated end time allows the user to plan their time more effectively :).

In the next article we'll look at indexed selects.

Labels: progress, select

Recently at Sprezz Towers, we had an interesting discussion on resolved selects, latent (or unresolved) selects, sizelocks and manipulating records while in a select list. Surprisingly, there was some confusion, even between ourselves. Once we hashed it all out, we thought it would be a good idea to post on the topic.

Types of Selects

As most of you know, there are two types of selects in the Revelation environment, latent and resolved. In resolved selects, the results are processed and stored in a temporary SYSLISTS (or LISTS) record. In latent selects, the record IDs are not returned until the READNEXT (or READNEXT BY) statement is executed. Resolved selects are initiated through the RLIST function or PERFORM statement. Latent selects are initiated through the Basic+ reduce subroutine and select statement.

What many people do not know is that inside each resolved select is a latent select waiting to get out. In order to generate the resolved select, the system must process the select through a latent list.

So, if you were to issue the following as a resolved select:

  SELECT SYSPROCS WITH TYPEID = 'STPROC' BY ENTITYID BY APPID

the system will basically execute code similar to this:


More or less...it's not perfect, but it's a pretty rough idea of what happens.

The system parses your sentence, generates a reduce, performs a select/readnext, stores all the returned keys, writes them off to a LISTS record (and updates the save select list queue) and then does a make.list so they are active and available.

In short, resolved selects are a giant shell around reduce/select. When you work with a resolved select, you're processing the list twice; once to resolve it, and once to process it.

It's always quicker to skip the shell and work with reduce/select yourself. However, as with many shelled functions, the system also handles some maintenance work for you.

In the next article, we'll talk about the maintenance work required to handle all this correctly.

Labels: latent, reduce, resolved, select