[Visual Basic 6] Calling DLL functions using CDECL calling convention

[vkamenar]

Officially, Visual Basic 6 supports calling external DLL functions and sub-procedures using only the stdcall calling convention.

For example, you can call the Win32 RtlMoveMemory API as follows:

Private Declare Sub CopyMemory Lib "kernel32" Alias "RtlMoveMemory" (hpvDest As Any, hpvSource As Any, ByVal cbCopy As Long)

Private Sub TestCopyMem()
   Dim my_array(20) As Byte
   CopyMemory ByVal VarPtr(my_array(0)), ByVal VarPtr(my_array(10)), 5
End Sub

Almost all Win32 API functions use the stdcall calling convention. However, there are a few cdecl API functions. For example, the apphelp.dll exports several cdecl functions:

• ApphelpDebugPrintf
• DlPrintf
• SE_ShimDPF

kernel32.dll and ntdll.dll also have some cdecl functions. There are also some DLLs that use mainly the cdecl calling convention. For example, openal32.dll, the 32-bit version of the OpenAL 3D audio API, is a cdecl-only DLL.

The ImpLib SDK allows calling cdecl functions from Visual Basic 6. There is a complete example showing how to use the OpenAL v1.0/v1.1 API in Visual Basic 6.0. This API uses the cdecl calling convention.

The first step is to create an import library for the DLL using the ImpLib SDK. For example, the following DEF-file specifies several cdecl functions available in the OpenAL API:

include 'implib.inc'

   vbimplib openal32, openal32.dll, CDECL, 1, alcOpenDevice
   vbimplib openal32, openal32.dll, CDECL, 1, alcCloseDevice
   vbimplib openal32, openal32.dll, CDECL, 2, alcCreateContext
   vbimplib openal32, openal32.dll, CDECL, 1, alcDestroyContext
   vbimplib openal32, openal32.dll, CDECL, 2, alDeleteSources
   vbimplib openal32, openal32.dll, CDECL, 0, alGetError

endlib

All functions are declared as cdecl in this example, but it's possible to combine cdecl functions and stdcall or other functions in the same import library.

For example, the functions alcOpenDevice and alcCloseDevice expect a single 32-bit parameter, while the functions alcCreateContext and alDeleteSources expect two 32-bit parameters. The function alGetError doesn't expect any parameters at all. It's important to specify the correct number of parameters, so that the ImpLib SDK generates the correct conversion code. Usually, the parameters on the stack use 32-bit alignment. Therefore, if there are 8- or 16-bit parameters (e.g. byte, short, char), they are also counted as 32-bit parameters. On the other hand, 64-bit parameters, such as double, are counted as two parameters because they take up twice as much space on the stack.

When compiling the import library, the ImpLib SDK generates a lightweight conversion code from cdecl to stdcall for compatibility with the compiler. The following image shows the conversion code disassembled from the executable file. For example, for alGetError there is no conversion because the function doesn't expect any parameters. That's why the function is called directly.

Let's save the DEF-file as openal32.def (feel free to use any other name, if you like) and compile the import library:

..\bin\fasm openal32.def openal32.lib
flat assembler  version 1.67.23  (786431 kilobytes memory)
DLL symbols indexed: 6
3 passes, 5107 bytes.

The import library openal32.lib must be copied to the Visual Basic 6 project directory.

The next step is to create a module file that contains the prototypes of these functions for Visual Basic 6. For example, let's create the module file openal32.bas with the following content and add this module to our Visual Basic 6 project:

Attribute VB_Name = "openal32"

Function alcOpenDevice(ByVal devicename As Long) As Long : End Function
Function alcCloseDevice(ByVal device As Long) As Boolean : End Function
Function alcCreateContext(ByVal device As Long, ByVal sttrlist As Long) As Long : End Function
Sub alcDestroyContext(ByVal context As Long) : End Sub
Sub alDeleteSources(ByVal n As Long, ByRef sources As Long) : End Sub
Function alGetError() As Long : End Function

The final step is to use the VB6LINK tool to substitute the module file openal32.bas with the import library openal32.lib. The module name must match the name of the import library, not including the extension. VB6LINK is an open-source tool included in the ImpLIb SDK. This tool is a wrapper for the MS linker for Visual Basic 6.

Copy VB6LINK.EXE to the Visual Basic 6 installation directory. The path is typically \Program Files\Microsoft Visual Studio\VB98\ (in Win32) or \Program Files (x86)\Microsoft Visual Studio\VB98\ (in Win64). Then rename the Microsoft linker LINK.EXE to LNK.EXE and rename the wrapper VB6LINK.EXE to LINK.EXE.

The wrapper substitutes the object file generated by the C2 compiler with the import library file of the same name, if any. For example, OPENAL32.BAS is compiled to OPENAL32.OBJ. Then, the wrapper substitutes OPENAL32.OBJ with OPENAL32.LIB to let the final executable link to OPENAL32.DLL. The substitution is done automatically every time you compile the Visual Basic 6 project from the IDE or from the command line.

If you want to remove the wrapper, just rename LNK.EXE back to LINK.EXE.