Howto loop through each element on a SafeArray with an unknown number of Dimensions?

eprogrammer — Sat, 22 Sep 2007 23:03:00 GMT

I thought that the subject ‘SafeArrays’ were really from the previous age and there was really nothing new about them, I could tell you , that was not already known.

So sit back and listen, what I am going to tell you, has been decided, in a dark hole, were some Microsoft Nerds must have thought that our programmers' live was too easy. So they modified the way the old variant arrays from the COM era were stored in memory.

Which Windows editions are involved in this modification? I don't know exactly. Anyway, Windows 64 bit edition (x64) really is subject to this, while Windows 2003 x86 (32 bit) is not. And don't worry too much, if you only use Ole Automation API calls, and not like I did, directly manipulated the SAFEARRAY storage area, your hard working code at your customers site, still should work

So, let’s figure what has changed by looking at the code below which very common.

SAFEARRAYBOUND bounds[1];

bounds.cElements = 10;
bounds.lLbound = 0;

SAFEARRAY *psa = SafeArrayCreate(VT_VARIANT, 1, &bounds);

This creates a variant array, that is compatible with scripting, similar to
Redim myArray(9) ' 0 to 9 is ten elements.

But before the array storage memory layout modification that I'm talking about,

LONG elSize = SafeArrayGetElementSize(psa); would return 16 (since sizeof(VARIANT) obviously equals that) but on Windows x64, this returns 24. Possibly, the alignment still is 16, but this makes me suspicious. Why did Microsoft change this? So I want to be sure it works independently of the Windows version or future features.

// code below gives an impression how to loop through an ‘unknonn dimension sized Safe Array, in this case the variant type is VT_VARIANT. Don’t use it. It’s faulty now on modern Windows systems.

LONG lElements = 0;

for (LONG x = 0; x < cDims; x++)

{

SafeArrayGetLBound(psa, x + 1, &lbound);

SafeArrayGetUBound(psa, x + 1, &ubound);

lElements += (ubound - lbound + 1);

}

hr = SafeArrayAccessData(psa, &psadata);
if (hr == S_OK)
{
                VARIANT *myvarray = static_cast<VARIANT *>(psadata);
                for (LONG els = 0; els < lElements && hr == S_OK; els++)
                {              // do something easy with the element
                               VARTYPE vt = myvarray[els].vt;
                }
                SafeArrayUnaccessData(psa);
}
The code above, would work to go through all elements for –any type of-SafeArray, VT_UI1, VT_BSTR, VT_VARIANT, because we assumed, for instance, that a VT_UI1, would be one byte long, isn’t it? And a BSTR Safe Array, would be a memory block, of BSTR pointers. And a VARIANT array, would have been a block of elements, consisting of VARIANTS.

So the code above, just worked, and looks in fact, pretty simple, fast and elegant.
Why so? Because it avoided the task to construct the indices one by one, and to fetch the element by using SafeArrayGetElement(psa, rgindices, outputvalue).

This one surely works, but has two important disadvantages,
1) when you deal with a known 3D or 2D sized-array, it's easy to fill the rgIndices in a loop, but if the dimensions are unknown, it would be required to iterate through the elements by other means. So you’re up to do some nasa stuff! J.
2) SafeArrayGetElement is relatively slow, since it copies the out value and you need to release the output (such as variants and strings). It’s like pumping around lots of memory, especially when the array is huge.

To get this working, I had to refresh my knowledge of some informatics school I did in the past, about simple math, about carry over on signed numbers. Since SafeArrays have different lowerbounds and upperbound ranges per dimension, you cannot simply multiply by a known row size, to initialize it. But anyway, to make a longer story shorter, and I might lack the correct choice of words, for things that I technically just ‘see’, I wrote the code for you. It’s pretty cool, I could not find such code through Google. J
Some tips. I decided to avoid reading the pointer to the safearray descriptor at all, because I saw some unexplainable swapping of rgsaBounds. Better use the API if there is one!

// assume psa to be of SAFEARRAY and this code is using ATL version 8

LONG cDims = SafeArrayGetDim(psa);

//we ‘rebuild’ the bounds by using the API, not by reading the psa structure directly to avoid unclear behaviour
CTempBuffer<SAFEARRAYBOUND> psaBound(cDims);

CTempBuffer<LONG> rgIndices(cDims);

LONG dimPointer = 0; // our dimension pointer, we go from left to right to build up the rgIndices

LONG currentEl = 0, ubound, lbound;

for (LONG x = 0; x < cDims; x++)

{

SafeArrayGetLBound(psa, x + 1, &lbound);

SafeArrayGetUBound(psa, x + 1, &ubound);

psaBound[x].cElements = ubound - lbound + 1;

psaBound[x].lLbound = rgIndices[x] = lbound;

}

// locking is not optional, it is needed.

SafeArrayLock(psa);

for(;;) //termination is decided within the loop

{

if (rgIndices[dimPointer] <

(LONG)psaBound[dimPointer].cElements + psaBound[dimPointer].lLbound)

{

VARIANT* pVar ;

// use the fast version instead of SafeArrayGetElement!

hr = SafeArrayPtrOfIndex(psa, rgIndices, (void**)&pVar);

if (FAILED(hr))

MYBAILOUT(hr);

rgIndices[dimPointer]++;

//this terminates the for as soon as we reached the last array element

if (++currentEl == lElements)

break;

}

// our carry on overflow stuff goes from left to right

else

{

while(rgIndices[++dimPointer]++ ==

(LONG)psaBound[dimPointer].cElements + psaBound[dimPointer].lLbound)

{

}

//reset previous cols to initial lowerbound from left to

// most right carry position

for (LONG z = 0; z < dimPointer; z++)

rgIndices = psaBound.lLbound;

// if carry has been done, we start counting on left again

dimPointer= 0;

}

To visualise this, I think it’s usefull to dry test this using a matrix.

Imagine, we have a script, that was created by your customer, in VBSCript and youre superpower, martian CPP needs to do something with this data.

So, the Customer code could be:

Redim MyArray(3,2)

How would our dynamic rgIndice be iterated?

rgIndices would follow this pattern.

Left dimension	Right dimension	Absolute element position
0	0	0
1	0	1
2	0	2
3	0	3
0	1	4
1	1	5
2	1	6
3	1	7
0	2	8
1	2	9
2	2	10
3	2	11

Of course, the algorithm, would go through any array, with any dimension.

Now have much fun with this code, if you needed it in your code. Don’t forget to add error handling which I left out to keep it short. And sure, you could write some C# stuff, for your COM interop to deal with array marshaling as well.

If this code was very useful for you, don’t forget to pay a visit to a component that uses this handy trick at http://www.nieropwebconsult.nl/asp_session_manager.htm

eprogrammer : Memory Storage Layout for Safe Arrays

Howto loop through each element on a SafeArray with an unknown number of Dimensions?