Explicit P/Invoke: Callbacks from native code

Posted September 6, 2009 by radiantway
Categories: Programming

Tags:

In the previous article I discussed how explicit p/invoke is used to call native code from the managed code. Here I ‘ll discuss what if native code needs to call managed code?
We can utilize Delegates for this purpose i.e. we can pass delegate as the parameter and then the native code can cast this as function pointer and store it for later use. This function pointer can be used to call managed code.
The following example demonstrates how to configure an unmanaged API to trigger a managed delegate.

[DllImport(@"path\Example.dll", ExactSpelling = true, CharSet = CharSet.Ansi)]
public static extern int CallBack(CallBackMethodDelegate delCallback, int n);

public delegate int CallBackMethodDelegate(int n);

public static int CallBack(int n)
{
return n / 2;
}

The following code shows how to register this delegate to native code and make it call back.

CallBackMethodDelegate delCallback = new CallBackMethodDelegate(CallBack);
int iResult = CallBack(delCallback,n);

The following is the native code for this example.

int CallBack(int (_stdcall *pFnCallBack)(int), int iParam)
{
int iProcessedInManagedValue = pFnCallBack(iParam);
return iProcessedInManagedValue;
}

The function pointer remains valid as long as delegate is inside managed code. So if your native application needs to store it for later use make sure corrosponding delegate exists for the same duration and does not get garbage collected.

Explicit P/Invoke: Memory management for return types and ref parameters

Posted September 6, 2009 by radiantway
Categories: Programming

Tags: ,

In one of my recent assignments at work I was to communicate with native code from my C# application. Micros0ft provides two kinds of platform invoke functionality i.e.

Implicit p/invoke: This is supported by VC++ only so managed C++ can call unmanaged code.

Explicit p/invoke: This is supported by .NET platform and any code written in C# or VB.NET can execute native code and vice versa.

Other than these, if native code is implemented as COM servers, .NET runtime provides COM Callable Wrappers to communicate with the managed code. Similarly it provides Runtime Callable Wrappers where managed code is exposed as COM objects for native code to communicate with the managed code.

In this article my focus will be on explicit p/invoke providing callback functionality and related memory issues.

To call unmanaged code from managed is pretty straight forward. All you have to do is to export the functions in the DLL. Then provide the declaration matching exact signature with Dllimport attribute and if custom structures or classes are being used as parameters provide their definition as well in the managed code.

For example if some dll Example.dll exports a function “int nStatus GetUpdate(char* strParam)”. To call this funciton in your .NET application declare it as follows

[Dllimport("Path\\Example.dll")]

public extern static int GetUpdate(MarshalAs(UnmanagedType.LPStr)] string strParam);

Then call it as follows

string strParam = "test data";

int iStatus = GetUpdate(strParam);

The native code would look like

int GetUpdate(char* strParam)

{

//Do some processing on strParam here...

// calculate status and return

return nStatus;

}

Lets involve custom structures first as value parameters and later on as ref parameter so that native application will allocate memory and managed code will consume.

[StructLayout(LayoutKind.Sequential)]
struct BridgeMember
{
public int nMember;
[MarshalAs(UnmanagedType.LPStr)]
public string strMember;
}
[StructLayout(LayoutKind.Sequential)]
struct BridgeData
{
public int nData;
public uint dwData;
[MarshalAs(UnmanagedType.LPStr)]
public string strData;
public eType eData;
public BridgeMember BMData;
}
enum eType { one = 1, two };

[DllImport(@"path\Example.dll", ExactSpelling = true, CharSet = CharSet.Ansi)]
public static extern int ProcessObjectByVal(BridgeData BInData);

Use it as follows
BridgeData oBridgeData = new BridgeData();
oBridgeData.BMData.nMember = 10;
oBridgeData.BMData.strMember = "From C# assembly";
oBridgeData.dwData = 20;
oBridgeData.eData = eType.one;
oBridgeData.nData = 30;
oBridgeData.strData = "From C#";

int n = ProcessObjectByVal(oBridgeData);

The native code will include the definitions of the structures exactly as above and the ProcessObjectByVal() will look like as follows


int ProcessObjectByVal(BridgeData BInData)
{
//get values from BInData and process...
return iSomeStatus;
}

Now if we want to pass BridgeData as ref or out parameter, for strings allocations has to be done using COM memory functions such as CoTaskMemAlloc. Because .NET runtime expect native code to allocate memory using CoTaskMemAlloc and it frees the pointers using CoTaskMemFree. If memory is allocated using other functions like new or malloc; the memory will not be freed or worse crash can occur as .NET runtime will try to free it using CoTaskMemFree in any case.
For details please this article on MSDN.
Following example illustrate the correct use of memory functions for p/invoke.
Native Code:

int ProcessObjectByRef(BridgeData* pBOutData)
{
pBOutData->BMData.nMember = 1;
int size = 400;
//pBOutData->BMData.strMember = new char[size]; //who will free it? using coTaskMemAlloc will fix this leak
pBOutData->BMData.strMember = (char*)CoTaskMemAlloc(size);
memset(pBOutData->BMData.strMember,0,size);
strcpy_s(pBOutData->BMData.strMember,size,"This is medium string");
pBOutData->dwData = 55;
pBOutData->eData = two;
pBOutData->nData = 14;
pBOutData->strData = (char*)CoTaskMemAlloc(20); //who will free it? will using coTaskMemAlloc fix this leak?
memset(pBOutData->strData,0,20);
strcpy_s(pBOutData->strData,20,"small string");
return pBOutData->nData;
}

Managed code:

[DllImport(@"path\Example.dll", ExactSpelling = true, CharSet = CharSet.Ansi)]
public static extern int ProcessObjectByRef(out BridgeData BInData);

BridgeData oBridgeDataOut;
n = ProcessObjectByRef(out oBridgeDataOut);
//use oBridgeDataOut here...

We just discussed out to process objects by ref where native code will allocate memory as required instead of passing String or StringBuilder with pre-allocated memory which might not be a good option for all cases where data is created by native application.

In the above example if IntPtr is used in place of attributed string, the managed code has to de-allocate the memory itself rather than relying on the Runtime for example

IntPtr IntPtrData;
String strData = Marshal.PtrToStringAuto(IntPtrData);
Marshal.FreeCoTaskMem(IntPtrData);

Desktop interaction from a Windows Service using DCOM in Vista

Posted April 17, 2009 by radiantway
Categories: Programming

Tags: , ,

In one of my recent work I was faced with a situation where desktop interaction was needed from a service. In legacy systems all you have to do was to enable the desktop interaction flag but this will not work in the future releases from the Microsoft.

In windows vista; services are run in a seperate session which is different than the interactive user sessions. It means services can only interact with session zero which elminiates the possibility of service interacting with any of the logged on users. Vista does provide option to enable this interaction but this is allowed only to support legacy services and will be excluded in the future service packs or new OS releases.  Here is the link for the white paper explaining it in detail.

So only way forward is to have a user process communicating with the service using Inter Process Communication methods which include MessgaeQueues, Pipes, Sockets and DCOM. I already had a DCOM ‘out of process’ component so I decided to add a new interface to it which will be used by the service to communicate with the desktop user. At startup service CoCreates the DCOM object and uses the returned Interface pointer to call interface methods.

This approach will work for all the scenarios except where DCOM component has singleton class factory i.e. it will have one instance alive at all times and will return the same object for each new CoCreate call by the client(s). The default behavior for the DCOM component is to acquire the identity of the launching application. If Service running under SYSTEM account instantiate it then this singleton DCOM component (.exe as out of proc in my case) will be running under SYSTEM account. After this if one of the user process needs to access this component and calls CoCreate. This will create another singleton instance running under CURRENT_USER. So component will no longer remain singleton. 

Add the following code snippet under NoRemove AppID block in the DCOM server .rgs file to change the identity to ‘interactive user’.

val RunAs = s 'Interactive User'

Following is the complete file to view as sample.

HKCR
{
NoRemove AppID
{
ForceRemove {2B38654A-8574-4676-AEBB-2E48E3945346} = s 'Name of the exe'
{
val RunAs = s 'Interactive User'
}

'%APPID%' = s 'Name of the exe'
'Name of the exe'
{
val AppID = s '%APPID%'
}
}
}

This way the DCOM server will run as current interactive user even if instantiated by the Service running under SYSTEM in session 0 and we can have a singleton instance independent of the client identities.

File Minifilter Part4: Sending huge data from kernel to user mode efficiently using variable-sized structures

Posted February 23, 2009 by radiantway
Categories: Programming

Tags: ,

Today I will discuss how to send huge chunks of data (usually received in write operations) from kernel mode file system filter to its user-mode counter part. The default implementation of microsft in its sample of file scanner minifilter can send few KB of data per call (fltsendmessage) . The reason is that the structure defined to send different parameters including data to be written can contain only handful of KB due to the small size of stack in the kernel mode.

There are many different structures defined internally in the windows code which can send more data at runtime than these are defined for at compile time. These structures are defined with a fixed header at begining and followed by a variable-sized array.  Then these structures are allocated with sizeof(structure) + number of bytes need(which are accesible by this last array later on). The old new thing is the one cool blog explaining about this variable-sized structure approach.

For kernel mode

typedef struct _SCANNER_INFO {
//other members above here ....
INT iLengthContents;
CHAR  szContents[1];
} SCANNER_INFO;


This is how you will allocate this structure

iNewSize = FIELD_OFFSET(SCANNER_INFO, szContents[iLengthRequired); //this macro FIELD_OFFSET is used to get the offset which in this case will be the total size of the structure

SCANNER_INFO* pScanInfo = ExAllocatePoolWithTag( NonPagedPool,iNewSize,'info' );

Then use this structure to send it to user-mode as follows

FltSendMessage(fitlerHandle, port, pScanInfo , iNewSize,pReplyBuf, &iReplySize,&lTimeout);

In the user mode the default sample contains the following the structure to receive data from kernel mode in overlapped way.

typedef struct _SCANNER_MESSAGE {
OVERLAPPED Ovlp;
//
//  Required structure header.
//
FILTER_MESSAGE_HEADER MessageHeader;
//
//  Private scanner-specific fields begin here.
//
SCANNER_INFO scanInfo;
}SCANNER_MESSAGE,* PSCANNER_MESSAGE;

Allocate and use this message struct as follows.

int iSize = sizeof(OVERLAPPED) + sizeof(FILTER_MESSAGE_HEADER)+ FIELD_OFFSET(SCANNER_INFO, szContents[SCANNER_MAX_PACKET_SIZE+1]);  //SCANNER_MAX_PACKET_SIZE is any required size
pMsg = (PSCANNER_MESSAGE)malloc(iSize);

//the actual buffer needed by getfilter callback, excluding overlapped size
iSize = sizeof(FILTER_MESSAGE_HEADER) + FIELD_OFFSET(SCANNER_INFO, szContents[SCANNER_MAX_PACKET_SIZE]);
memset( & pMsg ->Ovlp, 0, sizeof( OVERLAPPED ) );
hr = FilterGetMessage(iport, & pMsg ->MessageHeader,  iSize,  &msg->Ovlp );

This way we can overcome the limitation of sending more data then a kernel-mode stack can hold thus improving performance of the filter (used in encryption, virus scans etc) by manifolds.

File Minifilter part 3: How to know if current I/O request is from network in a file minifilter

Posted February 16, 2009 by radiantway
Categories: Programming

Tags: ,

This article is the part of series I started out to share my own difficulties faced while writing a file mini filter driver. Today I shall discuss how to find out if the current I/O request is comming from outside the local machine i.e. network.

One pretty easy way is to find the source of the current session. In a simple network there can be three sources.

  1. NTLMSSP (NT lan manager)
  2. SYSTEM (system user services and processes)
  3. USER32 ( all the prcesses running in the user space

The key here is to find this source if it is NTLM ( default lan manager on windows machine, some  network might use kerberos but its pretty easy to figure out once we get the source of the current I/O request’s session) then the current I/O request is comming from network e.g trying to access some shared folder. 

In case of SID we needed TOKEN_USER whereas here we are looking for  TOKEN_SOURCE  information associated with the current request.

Get token information as follows

PFLT_CALLBACK_DATA Data //comming from callback parameter

TOKEN_SOURCE **ppSource;
PACCESS_TOKEN pToken =  SeQuerySubjectContextToken(&(Data->Iopb->Parameters.Create.SecurityContext->AccessState->SubjectSecurityContext));
status = SeQueryInformationToken(pToken,TokenSource,ppSource);

TOKEN_SOURCE’s SourceName member contains the name of the source we are looking for, if it is NTLMSSP then the current request is comming from the network.

File Minifilter part 2: How to get SID and User name for I/O requests in File minifilter

Posted February 16, 2009 by radiantway
Categories: Programming

Tags: ,

Getting user name associated with the current I/O request can be obtained as shown below.

First of all we need a TOKEN_USER object from the PFLT_CALLBACK_DATA object.
There are two methods to achieve this.

A) From current thread(if impersonation is used) or process token

TOKEN_USER* TokenUser;
use ZwOpenThreadTokenEx() to get token handle.
HANDLE token;
if ((status=ZwOpenThreadTokenEx(NtCurrentThread(), GENERIC_READ, TRUE,OBJ_KERNEL_HANDLE , &token)) != STATUS_SUCCESS)
status=ZwOpenProcessTokenEx(NtCurrentProcess(), GENERIC_READ, OBJ_KERNEL_HANDLE,&token);

Get TokenUser from this token handle

ZwQueryInformationToken(token, TokenUser, NULL, 0, &len); //to get required length
if (!(*ppUser=ExAllocatePoolWithTag(NonPagedPool, len, 'ofni'))) //tagged memory allocation
return STATUS_NO_MEMORY;
if ((status=ZwQueryInformationToken(token, TokenUser, *ppUser, len, &len)) != STATUS_SUCCESS)
ExFreePoolWithTag(*ppUser, 'nacS');

B) From subject context present in the callback object

PFLT_CALLBACK_DATA Data; //passed from file filter module as parameter to callbacks
TOKEN_USER **ppUser; // token user to be used to extract SID
PACCESS_TOKEN pToken = SeQuerySubjectContextToken(&(Data->Iopb->Parameters.Create.SecurityContext->AccessState->SubjectSecurityContext));
*ppUser = NULL;
status = SeQueryInformationToken(pToken,TokenUser,ppUser);

Once TOKEN_USER structure is filled; following code can be used to generate a valid SID

TOKEN _USER* pUser; //it is the TOKEN_USER filled from one of the above mentioned methods

unsigned long dwStrSidSize=0;
SID* sid = (SID*)pUser->User.Sid;
dwStrSidSize = sprintf(strSID,"S-%u-",sid->Revision);
IdAuth = (sid->IdentifierAuthority.Value[5]) + (sid->IdentifierAuthority.Value[4] << 8 ) + (sid->IdentifierAuthority.Value[3] << 16) + (sid->IdentifierAuthority.Value[2] << 24);
dwStrSidSize += sprintf(strSID+strlen(strSID),"%u",IdAuth);
if (sid->SubAuthorityCount)
{
for (index = 0; index < sid->SubAuthorityCount;index++)
dwStrSidSize += sprintf(strSID+dwStrSidSize,"-%u",sid->SubAuthority[index]);
}

At the end free the allocated TOKEN USER as follows

ExFreePoolWithTag(pUser, 'ofni');

This way SID string can be achieved for current I/O request. This string can be cached (using SCANNER_STREAM_HANDLE_CONTEXT; will write about it in some other post) to be used in all the callbacks related to current I/O request.

This SID string can be passed to user mode code where user name, domain name, domain groups and other associated information can be retrieved.
For example:
use ConvertStringSidToSid to get SID structure.
LookupAccountSid can be used to retrieve account name of the user from this SID structure.

File MiniFilter Part1: Process name and ID in File Filter driver callbacks

Posted February 16, 2009 by radiantway
Categories: Programming

Tags: ,

Getting process name and process id is pretty easy in the file filter scanner callbacks.

Scanner callbacks contains a parameter PFLT_CALLBACK_DATA which has a pointer to thread which initiated the current I/O request.

Using this thread we can get process id in file filter as follows

PFLT_CALLBACK_DATA Data; //passed by file filter module
PEPROCESS objCurProcess = IoThreadToProcess( Data->Thread );
int iCurProcID    = PsGetProcessId(objCurProcess);

To get Process name

CHAR*  pStrProcessName = PsGetProcessImageFileName(objCurProcess);

 

So if you want to monitor I/O requests from some particular process you can do it on the basis of process name or process id. The following code will bypass all the subsequent File Filter Scanner callbacks from all the processes except explorer

if(_stricmp(pStrProcessName,"explorer.exe")==0 )
      return FLT_PREOP_SUCCESS_WITH_CALLBACK;
else
     return FLT_PREOP_SUCCESS_NO_CALLBACK;  // no more callbacks i.e. for postCreate, Pre/post Writes, Reads, setinfo and others.