[comtypes-users] Comtypes depends on numpy for wstring[] attributes

[Paul Moore]

I've just hit an issue using comtypes to interact with Virtualbox.
Here's the simplest test case I can create:

import comtypes.client
import comtypes.gen.VirtualBox

vbox = comtypes.client.CreateObject('VirtualBox.VirtualBox')
machine = vbox.FindMachine('Ahri')
adapter = machine.getNetworkAdapter(0)
nat = adapter.NATEngine
print(repr(nat.redirects))

This gets the redirects property from the NatEngine object, which is
documented as follows:

readonly attribute wstring[] redirects
Array of NAT port-forwarding rules in string representation, in the
following format: "name,protocol id,host ip,host port,guest ip,guest
port".

python .\test.py

Traceback (most recent call last):
File ".\test.py", line 8, in <module>
print(repr(nat.redirects))
File "C:\Work\Projects\vm\ve\lib\site-packages\comtypes\__init__.py",
line 279, in __getattr__
return getattr(self, fixed_name)
File "C:\Work\Projects\vm\ve\lib\site-packages\comtypes\safearray.py",
line 218, in __ctypes_from_outparam__
return self[0]
File "C:\Work\Projects\vm\ve\lib\site-packages\comtypes\safearray.py",
line 209, in __getitem__
return self.unpack()
File "C:\Work\Projects\vm\ve\lib\site-packages\comtypes\safearray.py",
line 234, in unpack
result = self._get_elements_raw(num_elements)
File "C:\Work\Projects\vm\ve\lib\site-packages\comtypes\safearray.py",
line 301, in _get_elements_raw
import numpy.ctypeslib
ImportError: No module named 'numpy'

As far as I can tell, comtypes is interpreting the array of wstring
objects as a 2-dimensional array, and as a result thinks it needs
numpy.

Two suggestions here:

1. If numpy isn't available, fall back to something core Python can
handle like a tuple of tuples.
2. In this particular case, the code should be returning a tuple of
(Unicode) strings.

The equivalent code works with pywin32, for what it's worth.

Paul

PS The website address and issue tracker mentioned on PyPI point to a
dead link - https://github.com/jaraco/comtypes/

[Chris Farrow]

Hi Paul,

Numpy should certainly be optional. I have created an issue for this:
https://github.com/enthought/comtypes/issues/55. We'll look into it.

The web address and issue tracker links have been fixed on the PyPI site.

Thanks,

Chris

[Paul Moore]

Post by Chris Farrow
https://github.com/enthought/comtypes/issues/55. We'll look into it.

I've just added a PR which fixes the BSTR issue for me, basically by
adding an explicit check for BSTR as a primitive type (like VARIANT).
It works for me, although it's a long time since I understood COM
refcounting semantics, so I may not have got the memory management
right.

Also, it does nothing for the "numpy should be optional" question, it
just avoids hitting it in this specific case.

Paul

PS The tests for safearray may need looking at as well - there's a
BSTR test that seems to rely on the current behaviour of using numpy,
if I'm reading it right - although I'm not clear why that would be
seen as correct behaviour.

[Chris Farrow]

Post by Paul Moore

Post by Chris Farrow
https://github.com/enthought/comtypes/issues/55. We'll look into it.

I've just added a PR which fixes the BSTR issue for me, basically by
adding an explicit check for BSTR as a primitive type (like VARIANT).
It works for me, although it's a long time since I understood COM
refcounting semantics, so I may not have got the memory management
right.

Thanks for the PR. I'll take a look as soon as I can.

Post by Paul Moore
Also, it does nothing for the "numpy should be optional" question, it
just avoids hitting it in this specific case.
Paul
PS The tests for safearray may need looking at as well - there's a
BSTR test that seems to rely on the current behaviour of using numpy,
if I'm reading it right - although I'm not clear why that would be
seen as correct behaviour.

I'll look into this as well. In the off chance that we're breaking
backwards compatibility, then we may have to consider another approach -
something like try numpy first, and fall back to tuple if that fails.

Cheers,

Chris

[Paul Moore]

I'll look into this as well. In the off chance that we're breaking backwards
compatibility, then we may have to consider another approach - something
like try numpy first, and fall back to tuple if that fails.

By the way, the test suite doesn't work for me. Some issues:

1. Needs to be run as admin to register a DLL (otherwise it gives an
obscure "Access is denied" error)
2. Doesn't unregister the DLL when it's done.
3. Lots of errors from numpy not being there (including a lot for
isdatetime64 which may be a numpy issue not sure).

I'll try to get a copy of numpy (I need a wheel so I can install it in
a virtualenv, I think there are some around somewhere) and have
another go.

Paul

[Paul Moore]

Post by Paul Moore
I'll try to get a copy of numpy (I need a wheel so I can install it in
a virtualenv, I think there are some around somewhere) and have
another go.

Lots better. There's a C runtime error which I suspect is because
Python 3.3 and Python 2.7 use different C versions. Otherwise it does
look like it was numpy dependencies.

Paul