|
|||||||
We recently came across a situation that we have literally never encountered in our decades of working with Linear Hash. Basically a FIXLH operation was interrupted mid process and hi-jinks ensued. The resultant mess took a while to sort out but our findings may prove useful if you ever find yourself in a similar situation. This specific issue was encountered on an AREV 3.12 system but the logic for FIXLH remains roughly the same so the information remains applicable. Basically, the FIXLH operation was interrupted about 10% of the way through processing (the NTVDM abended). A quick check of the table header revealed that it believed itself to be a clean empty table with no rows. Yet listing the table produced a set of row ids after an initial pause while nothing was apparently happening. What had happened was that the table header had been reset to that of an empty table, and the first 10% or so of the primary frames had been blanked. This caused us to investigate the processing undertaken by FIXLH and we can confirm that it is as follows: Create and/or clear the temporary dumpfix table grab frame zero group write rows in group to temporary dumpfix table set frame zero header to that of an empty table loop grab next group write rows in group to temporary dumpfix table wipe primary frame to null until no groups left repeat copy rows in temporary table back to source table So if the operation is interrupted, some of the data will be in the source table and some will be in the temporary dump fix table. As an additional caveat, note that if the rows in the temporary dump fix table are not stored off elsewhere, they will be lost next time FIXLH is run. Since the advent of the UD we've never had to actually fix a GFE for quite some time, so this was an eye-opening operation! Backups are your friend... Recently at a client site, we encountered an error that was a little more baffling than normal. Under very specific circumstances, RLIST would bail with no results. Eventually, we had a duplicatable test sentence which was basically 1. PROCMON and other useful tools are available as part of the Sysinternals suite of development tools, which no programmer should be without.
Those of you who've been in the industry for a while will remember the smugness with which we treated the Y2K issues the rest of the world were experiencing. It didn't affect us, we stored our dates in internal numerical format. I'm sure we all have favourite anecdotes. Mine was being called into the Press Office of a major Government department to certificate their AREV tracking system. I sat down and gave the system a cursory glance. "Ermmm - it doesn't seem to use dates?". "No, it doesn't". "Well, here's your certificate"... turns out they couldn't get the budget for system tweaks but they could for Y2K compliance. I spent the rest of the day implementing their desired changes. Anyway, this week wiped the smile off of our collective faces (if you'd ignored the advice in KB 42 thirty or so years ago - which to be honest you could be forgiven for) with the advent of our very own variant - we'll call it the 20100 bug, although it's not a bug, it's an unfortunate feature. Users began reporting that date searches were failing for values after the 10th of January 2023. At first we couldn't see an obvious reason. We built a database containing date values going back to the previous century with five rows per date and indexed it. We wrote a test program and ran it RUN ZZ_POP 10/01/2023 RUN ZZ_POP 11/01/2023 RUN ZZ_POP 20099 What we've fallen foul of is discussed in the KB reference earlier. Basically BTREE.EXTRACT takes what it is given and tries to ICONV it to use for the look up. If it can't ICONV it (20099 isn't a viable date) it uses the value passed. If it CAN ICONV it, is uses the ICONVed value. And guess what? 20100 ICONVs to the 2nd of January 2000 as you can see above. Of course, this won't be the first time this has happened in the wild - for example looking for the 7th of December 1998 using 11299 would have returned the 1st of December 1999 and so on. Note that the same issue will be experienced when using internal date formats with RLIST statements. The solution? When calling BTREE.EXTRACT use EXTERNAL values as it will try and ICONV the data before using it. AddendumWe've been asked to share the code we used to locate btree.extracts in client systems to enable a manual inspection to determine possible failure points. This is a rough and dirty hack which met our requirements. Feel free to tailor to your own requirements. 0001 Subroutine zz_find_Btree_Extract( void )
0002 /* 0003 Author AMcA 0004 Date Jan 2022 0005 Purpose Quick hack to help identify system usage of btree.extract 0006 Provided as is with no warranty 0007 */ 0008 0009 Gosub initialise 0010 Gosub process 0011 0012 Return 0013 0014 initialise: 0015 0016 filesToCheck = "SYSPROCS,SYSREPOSEVENTS,SYSTABLES" 0017 columnsToCheck = ",,8" 0018 0019 loopCtr = Dcount(filesToCheck, ",") 0020 resultString = "" 0021 0022 Return 0023 0024 process: 0025 0026 For loopPtr = 1 To loopCtr 0027 file = Field( filesToCheck, ",", loopPtr ) 0028 column = Field( columnsToCheck, ",", loopPtr ) 0029 0030 resultString := file : \0D0A\ 0031 0032 if file = "SYSTABLES" Then 0033 starting = "%" 0034 Gosub processSysTables 0035 End Else 0036 starting = "@" 0037 Gosub processRest 0038 End 0039 0040 resultString := \0D0A0D0A\ 0041 0042 Next 0043 0044 Oswrite resultString On "ZZ_BE.TXT" 0045 Call Set_Property("CLIPBOARD", "TEXT", resultString ) 0046 0047 Return 0048 0049 processRest: 0050 Open file To v Then 0051 Gosub processV 0052 End Else 0053 Call FSMsg() 0054 End 0055 Return 0056 0057 processV: 0058 0059 Select v 0060 eof = 0 0061 Loop 0062 Readnext id Else eof = 1 0063 Until eof Do 0064 If id[1, 1] = starting else 0065 Read row From v, id Then 0066 ptr = 1 0067 If column Then 0068 saveRow = row 0069 row = row< column > 0070 Convert @Vm To @Fm In row 0071 end 0072 lenRow = Len(row) 0073 lineNo = 0 0074 If lenRow > 0 then 0075 Loop 0076 nextline = row[ptr, @Fm] 0077 ptr = Col2() + 1 0078 lineno += 1 0079 there = IndexC( nextLine, "btree.extract(", 1) 0080 If there Then 0081 variable = Trim( nextLine[ there + 14, ","] ) 0082 If variable[1, 1] = "'" Or variable[1, 1] = '"' Then 0083 // passing literals - check the line 0084 resultString := file : \09\ : id : \09\ : lineNo : \09\ : nextLine : \0D0A\ 0085 End else 0086 // ok we're now going to work back through the code until we find = for our var 0087 tempLineNo = lineNo 0088 Loop 0089 tempLineNo -= 1 0090 line = row< tempLineNo > 0091 If Trimf( line )[1, 1] = "*" Then 0092 line = "" 0093 End 0094 Convert " " To "" In line 0095 line = " " : line 0096 Until IndexC( line, " " : variable : "=", 1) 0097 Until tempLineNo = 0 0098 Repeat 0099 If tempLineNo = 0 then 0100 resultString := file : \09\ : id : \09\ : lineNo : \09\ : "Not found " : row< LineNo > : \0D0A\ 0101 End else 0102 resultString := file: \09\ : id : \09\ : lineNo : \09\ : row< tempLineNo > : \0D0A\ 0103 end 0104 end 0105 End 0106 While ptr < lenRow 0107 0108 Repeat 0109 End 0110 End Else 0111 end 0112 Call send_info( file : " " : id ) 0113 end 0114 repeat 0115 return 0116 0117 processSystables: 0118 0119 Open file To v Then 0120 Select v 0121 saveV = v 0122 eof = 0 0123 dictCtr = 0 0124 0125 Loop 0126 Readnext id Else eof = 1 0127 Until eof Do 0128 If ID[1, 5] = "DICT." Then 0129 Open id To v Then 0130 0131 file = id 0132 0133 Call push.Select( v1, v2, v3, v4 ) 0134 Gosub processV 0135 Call pop.Select( v1, v2, v3, v4 ) 0136 eof = 0 0137 0138 End Else 0139 * Call FSMsg() 0140 End 0141 v = saveV 0142 end 0143 Repeat 0144 End Else 0145 Call FSMsg() 0146 End 0147 0148 return Tabbed interfaces provide a much loved way of fitting lots of information onto a single entry form. The introduction of multi page forms and the improved integration of page controlling made this a much easier exercise - each tab could be a page of the form and we could simply move between pages on a tab click. In fact a quick event was added to make this even easier. But what has not been possible until now, without a deal of smoke and mirrors, is to have one or more tab controls embedded within a page on a single or multi-page form and have them only appear on a specific page. The only way to accomplish this would be to have a set of controls per tab, all within the tab boundaries, and make them visible or invisible depending on the currently clicked tab. This makes form maintenance incredibly tedious and code overly complex. Before OI10 there was no real concept of parent/child relationships in OI - other than that forms and group boxes owned all of the controls within their boundaries. This led to many issues, not least with group boxes and tab controls. With the advent of OI10 this has changed. OI now allows the implementation of, and respects, parent/child relationships. The biggest example of this is the introduction of a "Panel" container type. We are going to use the "Panel" (not the "Simple Panel") to enable tabs within a form. But first, what's the difference between a "Panel" and a Simple Panel? A Panel is simply a container that can be placed anywhere on the screen so that when controls are pasted into the Panel they become children of the Panel, not of the Window. So move the Panel and the controls owned by the Panel move with it. Don't be too put out by the "Simple" part of the description, pretty much the only difference between a Panel and a Simple Panel has to do with paging. So looking at Simple Panel/Panel properties in the Form Designer we see this : PageCount and the only property unique to Simple Panels is the DummyCaption property. So, to implement a tab within a form page, we can now combine the Tab Control, the Panel Control and one simple Quick Event to achieve the desired results. Let's start with a simple one page entry screen. Select "Panel" from the Containers and position it within the tab control - using Right-Click and drag to size it appropriately. (See Addendum for other keystroke combinations). There are no hard and fast rules about this but I tend to leave an 8 pixel border from the tab control to make it easier to edit in the form designer. (As an aside, if you right-click on the Window you can un-anchor it to allow moving it wherever you want.) Tell the Panel that it has three pages and label the tab accordingly (after setting the tab to three values) : Select the Panel and add in the controls for the first page. Now select the Panel and use the spin control to move to page 2 and add in the controls for the second page. Note that the spin control applies to whichever control is both currently selected and supports paging - namely Windows and Panels. And finally the third page. Finally we need to add a Quick Event on the Tab click. This needs to get the VALUE Property of the Tab control, and set the CURRENTPAGE Property of the Panel. and Bob, as they say, is your uncle... Actually a quick revisit - internal discussion at Sprezz Towers brought up the fact that the article missed a fairly big point (because the author ass-u-med it was obvious which it obviously isn't :)). Panels can host panels that can host panels - as far as you are practically likely to go. So in the screen below we see multiple panels in action. Addendum - Mouse Use In Form DesignerWhen in Draw Mode: Left Click 1) Left-click to drop a default-size control on a the form 2) Left-click and drag to draw a control on the form 3) Ctrl-left-click to drop a default size control on a container control 4) Ctrl-left-click and drag to draw a control on a container control. Right Click 1) Right-click and drag to draw a control on the form 2) Right-click and drag to draw a control on a container control. When in Normal Mode: Left Click ========== 1) Left-click on the form to unselect all controls 1) Left-click on an unselected control to select it 2) Left-click and drag around/across a group of unselected controls on the form to select them 3) Left-click and drag on an unselected or selected control to move it 4) Left-click and drag around a group of selected controls to move them 5) Ctrl-left-click on a selected container control and drag around/across unselected child controls to select them 6) Ctrl-left-click on a selected container control and drag around/across unselected child controls to select them 7) Shift-left-click to add an unselected control to a group of selected controls with the same parent 8) Shift-left-click on a group-selected control to remove it from a group of selected controls Right Click =========== 1) Right-click on the form to display it's context menu 2) Right-click on an unselected control to select it and display it's context menu 3) Right-click on a selected control to display it's context menu 4) Right-click on a group of selected controls to display the group-selection context menu 5) Right-click and drag on the form to draw the most recently added type. 6) Right-click and drag on an selected container control to draw the most recently added type and add it as a child. 7) Right-click and drag on an unselected container to draw the most recently added type and add it as a child. 8) Shift-right-click on the form to drop the the most recently added type with a default size. 9) Shift-right-click on an selected container control to drop the the most recently added type with a default size and add it as a child. 10) Shift-right-click on an unselected container control to drop the the most recently added type with a default size and add it as a child. Recently, a client was a little late in applying their authorisation code to their copy of OI10, and attempts to access OI were met with error messages "PSSI.66: ???". This seemed a strange message and at first we suspected that it wasn't even an OI message. However chatting to the Rev developers, it seems that an insert (PS_RESOURCES) was missing from SYSENV and that it should in fact have had the following values /////////////////////////////////////////////////////////////////////////////// //[PS_SYSINTIALIZE]//////////////////////////////////////////////////////////// PSSI.00: No entry window defined for the %1% application. So that clarified that issue. The issue that we were left with was that normally in OI10, reauthorisation is done from within the product - and if you can't log in... you can connect the dots. So, we went on search for a reauthorisation executable in the OI directory and not surprisingly found revauth.exe. This is a small executable, and clicking it seemed to do nothing. Until we realised that it was likely a command line executable. Invoking powershell, we moved to the UNC containing revauth.exe and executed it with a /?. These were the results. PS Microsoft.PowerShell.Core\FileSystem::\\MyServer\Shares\Sprezz\programs\OINSIGHT_10> .\revauth.exe /? revauth <featurename> <authcode> <filename> values for <featurename>: OI : OpenInsight So we instructed the client to run the following command revauth OI AuthCode revengine.lic and lo and behold, the system worked again! As an aside, the licensing information is no longer stored encrypted in a DLL but rather is stored in the revengine.lic file where it can be easily viewed - a modified example is shown below. <OI> Of course, you can't alter any of these values in notepad as they are actually encoded into the signature, but nice try! And for readers who aren't British and of a certain age... One of the intermittent problems we face when developing with OI 10 is knowing about the new properties and features that we now have access to. The pace of development is so rapid that the documentation just doesn't keep up. There's a fine source of documentation at revDevX.com along with multiple blog posts at the same site but sometimes the Lord helps those who help themselves. With this in mind, we'll use a real world example to illustrate how much easier life can be when we have access to such new features. We recently wanted to have a radio button contract or expand based upon a control record. We knew how we would have accomplished this in the past - we'd grab the button control object, destroy it, modify it then recreate it. That's all well and good, it works and the control stays at the right place in the tab order - but it can change the Z-Order of the control which isn't always desirable. Naturally we wondered if there was a better way in 10. Of course there is, or we wouldn't be making this blog post. Taking a simple example where we want to alter the available buttons based on the membership type, we'd have one set of radio buttons for the member type and another for the letter type. An existing member wouldn't need a Welcome Letter so clicking on "Existing Member" has to do this i.e. the "Welcome Letter" option is removed. We all like to boast that OpenInsight is written in OpenInsight but the implications of this can be missed in the excitement of the new environment. We're so used to OI not exposing all of a control's properties that we lose track of the possibility that what we need may be staring us right in the face. The simple fact is, that if something exists in the property panel in the form designer for a control, then it is an exposed property. So let's look at the property panel for a radio button. and what do we see as the fourth property? Buttons. Clicking on the options button displays a dialog containing the text and values for the radio button :- As we've telegraphed this far in advance you can probably guess what this means - radio buttons expose a BUTTONS property. It consists of two columns, both multivalued. The first is the label description and the second is the associated code.So now, to accomplish the above goal all we have to do is manipulate this property. Here's a simple code snippet to do this :- If you're not familiar with the syntax being used for getting and setting properties check out the blog post here. We'd encourage you to take a look around the property panels to see what other goodies you can find! We've been doing a lot of stress testing of LH at Sprezz Towers to help a client with speed issues on large tables. As part of that we've created some 60 million row tables to play with combinations and permutations of differing indexing setups. In so doing, we came across a situation where the index sort would get to 80% and then just exit with an FS220 error. (FS_REL_NEED_REBUILD_ERR$). Now to be honest, this was confusing to us as there wasn't a relational index on the table. So speaking to the good folks at Rev, it was clarified that this was the wrong error but for fairly obvious reasons we were the first to notice. After discussing further with them we came to the conclusion that it must be related to disk space. So began an increasingly frustrating attempt to free up disk space (why oh why did I format my USB sticks with FAT32 and its 4GB file size limit?). Subdirectories were zipped, old files deleted until - finally we had the 40GB or so free that we needed. We tried again - and again around the 80% mark the index rebuild just stopped. It didn't crash, it just stopped - having cleared out the %ALL.IDS% token in the ! file. As V119 was suspected to be the culprit, we redirected the temporary sort file in the environment to the same disk as the data tables so that we could keep an eye on what happened. It seems that the largest sort file that can be created is 2GB and that attempts to exceed this will fail. Note that this applies to both IDX_SETS1 and 2. This is a limitation of a 32 bit OS rather than OpenInsight, so can not easily be addressed in version 9.x. Now it should be pointed out that this is a particularly extreme situation, as the row ids in this large table are 5 part and quite long. With a more normal key structure than the long complex one we were using, it is unlikely that this limit would be breached without having hundreds of millions of rows in the table... but it does mean that if you need to rebuild such large indexes you need to undertake the task in OI 10.1 where the issue has been resolved. Note also that this doesn't affect day to day use of the indexed table, normal additions and deletions will still update the index - this issue only affects a rebuild. As a side note - if you DO want to do this in 10.x there is a major caveat. Indexing has been rewritten for 10 and works in a different way than 9. This means that out of the box, OI 9 and OI 10 have different indexing routines and are not compatible. Fortunately Rev have provided a way to deal with this. All that we need to do is edit or create a record in SYSENV called CFG_RTI_UPDATE_INDEX. Set line 1 to RTI_UPDATE_INDEX_90 then save it, exit OI and restart. This will force 10 to use 9 indexing logic. We've been working a lot with indexes on large tables in 10 and we'd recommend using this setting in any case if you're working with large data volumes. By large tables, we are talking tens of millions of rows. For the avoidance of doubt - if you are sharing data between OI 9 and OI 10 you MUST do this or you will experience issues.
|
|||||||
| |||||||