UTF8 - The BRemove statement

Recent changes to the "[]" operators in OpenInsight 9.2 have resulted in substantial performance improvements to UTF8 mode string handling. This post highlights another such enhancement introduced in 9.2 to help bring UTF8 mode applications up to the standard of their ANSI counterparts.

Consider the Loop/Remove construct below:

0001  /*
0002     Example showing standard loop/remove construct used
0003     to parse dynamic arrays at high speed
0004  */
0005  
0006     mark = 1
0007     pos  = 1 ; * // This is the CHARACTER position
0008     Loop
0009        Remove nextVal From dynArray At pos Setting mark
0010        
0011        // Process nextVal...
0012        
0013     While mark
0014     Repeat 

This is a common way to efficiently parse dynamic arrays in Basic+, but just like the normal "[]" operators it suffers from a severe performance degradation in UTF8 mode due to the need to find the byte offset of a character when given the position.

To alleviate this Revelation have introduced the BRemove statement - this operates in exactly the same fashion as the normal Remove statement, but the index variable used in BRemove refers to a byte offset rather than a character position.

Here is the same example rewritten to use BRemove:

0001  /*
0002     Example showing UTF8-friendly loop/remove construct used
0003     to parse dynamic arrays at high speed
0004  */
0005  
0006     mark = 1
0007     pos  = 1 ; * // This is the BYTE offset
0008     Loop
0009        BRemove nextVal From dynArray At pos Setting mark
0010        
0011        // Process nextVal...
0012        
0013     While mark
0014     Repeat 

As you can see it's a simple change and one worth making - using BRemove in your UTF8 applications will ensure that your dynamic array parsing remains fast and efficient.