Of late at Sprezz Towers we've been all about taking apart Linear Hash frame headers to diagnose some, frankly bizarre, issues being experienced by one of our clients. Our utilities were working fine on "old style" Linear Hash files where the maximum frame size was 64K and the maximum file size was 4GB but on the newer UD3+ files (or "type 3" as they're referred to) we just didn't seem to be getting the consistent results we were expecting.

As an example consider this section of code for examining the "Alpha Space Usage" - the total bytes of data stored in the Primary frames :-



So this snippet takes the 8 bytes that comprise the Alpha value and converts them into a very large integer.

When the Alpha was a value of 3,349,830,462 the MX conversion returned C7AA5B3E which is indeed correct. The issue arose when the Alpha had increased to 7,644,806,759, then the MX conversion returned an erroneous value and got the hex conversion wrong.

Those of you with knowledge of typed languages can probably see where this is going! As OpenInsight is a 32 bit app, the largest value that can be represented in an unsigned integer is 4,294,967,295, which the 3,349,830,462 is less than - 7,644,806,759 however is over this limit so the MX conversion failed.

Fortunately for us the solution was simple (once we realised that something needed solving. The MX conversion works off an integer, but the HEX conversion works off an ASCII string. So all we had to do was modify our code to use this, and convert the actual string value not the number value as below :-



and all was once again right with the world with the HEX conversion returning 677EAAC701000000.

Finally we were able to report on the corrupted file accurately.

Frame length	1,634,013,184	00 10 65 61
Modulo	2,342,159	0F BD 23 00
Alpha	7,644,806,759	67 7E AA C7 01 00 00 00
Threshold	80
Sizelock	0
Row Count	20,379,590	C6 F7 36 01
Alpha%	.00

Header	32 0 0 0 0 0 0 0 0 15 189 35 0 0 16 103 126 170 199 204 0 0 198 247 54 1 101 97 1 0 0 0
Header	20 0 0 0 0 0 0 0 0 F BD 23 0 0 10 67 7E AA C7 CC 0 0 C6 F7 36 1 65 61 1 0 0 0

CAVEAT
It should be noted that the code snippet above is not UTF-8 safe. We knew that the data we were working on was purely ANSI and had customised one of our routines accordingly. The square bracket operators, CHAR and SEQ are generally not safe when processing binary data so unless you're 100% sure you're in ANSI mode avoid them. In this case we should have been using GetBinaryValue.