Array of variable length strings

Currently only arrays of fixed-length strings are supported by JADE.

Array of variable length strings would be particularly useful when doing things like CSV imports, where each line is split into a string array for processing. With an array of fixed-length strings, the length of the strings need to be set high enough for any eventuality. We therefore have developers using arrays of maximum sized strings (length of 15999). This results in a huge waste of resources.

I realize that for performance reasons, in a lot of cases arrays of variable length strings are not appropriate, but there are times when they are appropriate and it would be nice to have the option.

I submitted this as an NFS in 2006 (#41070).

John Munro
May 1 2019
Shipped (2020)

Root Schema

HistoricalComments

John Beaufoy 11/10/2017 3:26:16 PM
Had a similar problem - built a VariableStringArray class that utilises a JadeBytes structure with an array to record offsets for each index. If you track me down @ NWI I'd be happy to forward on the code. I should stress that this solution is less than ideal - effectively a hack - so having an inbuilt solution would be cleaner, more accessible and more performant.

Comments (2)
Votes (14)

Attach files

Enter a subject

Nick Scott commented

August 31, 2020 20:50

All instances of arrays of Binary, String and StringUtf8 will have a new internal format. Collection blocks are smaller, resulting in improved performance, reduced object cache footprint and database file size. The bound on the maximum length of entries in the Array classes BinaryArray, StringArray and StringUtf8Array will be increased to the upper bound for the membership of the array: BinaryArray from 128 to 16000 Bytes, StringArray from 62 to 15999 Characters, HugeStringArray from 2047 to 15999 Characters and StringUtf8Array from 30 to 8000 StringUtf8 Characters. HugeStringArray effectively becomes an alias for StringArray. The upper bound on user defined Array subclasses will not be changed. When the database is upgraded from 2018 any persistent instances will be converted to the new format. No application changes will be required to take advantage of the improvements with the new implementation.

×
Attachments Open full size
Chris Power commented

August 31, 2020 18:50

What are the trade-offs of converting uses of fixed-length string arrays like HugeStringArray to the new type? I'm thinking that most uses have a large number of small strings but use, say, HugeStringArray to cater for odd cases. Is there a formula for the persistent storage requirements for old and new string arrays? Is the performance roughly the same?

×
Attachments Open full size

JADE Environment Development Ideas

What's new in the upcoming JADE release?

IMPROVED DEVELOPER EFFICIENCY. ENHANCED SECURITY. SMOOTHER INTEGRATION