Home page Home page Home page Home page
Pixel
Pixel Header R1 C1 Pixel
Pixel Header R2 C1 Pixel
Pixel Header R3 C1 Pixel
Pixel
By Sprezz | Tuesday, 26 April 2016 15:23 | 0 Comments
Of late at Sprezz Towers we've been all about taking apart Linear Hash frame headers to diagnose some, frankly bizarre, issues being experienced by one of our clients. Our utilities were working fine on "old style" Linear Hash files where the maximum frame size was 64K and the maximum file size was 4GB but on the newer UD3+ files (or "type 3" as they're referred to) we just didn't seem to be getting the consistent results we were expecting.

As an example consider this section of code for examining the "Alpha Space Usage" - the total bytes of data stored in the Primary frames :-



  lo_alpha  = file[alphaPtr$, 4]
  hi_alpha  = file[alphaPtrHi$, 4]

  If hiBitSet Then lo_alpha := hi_alpha

  value = lo_alpha
  Gosub Convert
  lo_alpha = value

  hlo_Alpha = Oconv( lo_alpha, "MX")
    

So this snippet takes the 8 bytes that comprise the Alpha value and converts them into a very large integer.

When the Alpha was a value of 3,349,830,462 the MX conversion returned C7AA5B3E which is indeed correct. The issue arose when the Alpha had increased to 7,644,806,759, then the MX conversion returned an erroneous value and got the hex conversion wrong.

Those of you with knowledge of typed languages can probably see where this is going! As OpenInsight is a 32 bit app, the largest value that can be represented in an unsigned integer is 4,294,967,295, which the 3,349,830,462 is less than - 7,644,806,759 however is over this limit so the MX conversion failed.

Fortunately for us the solution was simple (once we realised that something needed solving. The MX conversion works off an integer, but the HEX conversion works off an ASCII string. So all we had to do was modify our code to use this, and convert the actual string value not the number value as below :-



  lo_alpha  = file[alphaPtr$, 4]
  hi_alpha  = file[alphaPtrHi$, 4]

  If hiBitSet Then lo_alpha := hi_alpha
  hlo_Alpha = Oconv( lo_alpha, "HEX")

  value = lo_alpha
  Gosub Convert
  lo_alpha = value
    

and all was once again right with the world with the HEX conversion returning 677EAAC701000000.

Finally we were able to report on the corrupted file accurately.

Frame length 1,634,013,184 00 10 65 61
Modulo 2,342,159 0F BD 23 00
Alpha 7,644,806,759 67 7E AA C7 01 00 00 00
Threshold 80
Sizelock 0
Row Count 20,379,590 C6 F7 36 01
Alpha% .00

Header 32 0 0 0 0 0 0 0 0 15 189 35 0 0 16 103 126 170 199 204 0 0 198 247 54 1 101 97 1 0 0 0
Header 20 0 0 0 0 0 0 0 0 F BD 23 0 0 10 67 7E AA C7 CC 0 0 C6 F7 36 1 65 61 1 0 0 0

CAVEAT
It should be noted that the code snippet above is not UTF-8 safe. We knew that the data we were working on was purely ANSI and had customised one of our routines accordingly. The square bracket operators, CHAR and SEQ are generally not safe when processing binary data so unless you're 100% sure you're in ANSI mode avoid them. In this case we should have been using GetBinaryValue.

0 Comments:

Post a comment

Subscribe to Post Comments [Atom]



<< Home

Pixel
Pixel Footer R1 C1 Pixel
Pixel
Pixel
Pixel