Tutorial 13: Memory Mapped Files

I'll show you what memory mapped files are and how to use them to your advantages. Using a memory mapped file is quite easy as you'll see in this tutorial.
 

Theory:

If you examine the example in the previous tutorial closely, you'll find that it has a serious shortcoming: what if the file you want to read is larger than the allocated memory block? or what if the string you want to search for is cut off in half at the end of the memory block? The traditional answer for the first question is that you should repeatedly read in the data from the file until the end of file is encountered. The answer to the second question is that you should prepare for the special case at the end of the memory block. This is called a boundary value problem. It presents headaches to programmers and causes innumerable bugs.
It would be nice if we can allocate a very large block of memory, enough to store the whole file but our program would be a resource hog. File mapping to the rescue. By using file mapping, you can think of the whole file as being already loaded into memory and you can use a memory pointer to read or write data from the file. As easy as that. No need to use memory API functions and separate File I/O API functions anymore, they are one and the same under file mapping. File mapping is also used as a means to share data between processes. Using file mapping in this way, there's no actual file involved. It's more like a reserved memory block that every process can *see*. But sharing data between processes is a delicate subject, not to be treated lightly. You have to implement process and thread synchronization else your applications will crash in very short order.
We will not touch the subject of file mapping as a means to create a shared memory region in this tutorial. We'll concentrate on how to use file mapping as a means to "map" a file into memory. In fact, the PE loader uses file mapping to load executable files into memory. It is very convenient since only the necessary portions can be selectively read from the file on the disk. Under Win32, you should use file mapping as much as possible.
There are some limitation to file mapping though. Once you create a memory mapped file, its size cannot be changed during that session. So file mapping is great for read-only files or file operations that don't affect the file size. That doesn't mean that you cannot use file mapping if you want to increase the file size. You can estimate the new size and create the memory mapped file based on the new size and the file will grow to that size. It's just inconvenient, that's all.
Enough for the explanation. Let's dive into implementation of file mapping. In order to use file mapping, these steps must be performed:
  1. call CreateFile to open the file you want to map.
  2. call CreateFileMapping with the file handle returned by CreateFile as one of its parameter. This function creates a file mapping object from the file opened by CreateFile.
  3. call MapViewOfFile to map a selected file region or the whole file to memory. This function returns a pointer to the first byte of the mapped file region.
  4. Use the pointer to read or write the file
  5. call UnmapViewOfFile to unmap the file.
  6. call CloseHandle with the handle to the mapped file as the parameter to close the mapped file.
  7. call CloseHandle again this time with the file handle returned by CreateFile to close the actual file.

Example:

The program listed below lets you open a file via an open file dialog box. It opens the file using file mapping, if it's successful, the window caption is changed to the name of the opened file. You can save the file in another name by select File/Save as menuitem. The program will copy the whole content of the opened file to the new file. Note that you don't have to call GlobalAlloc to allocate a memory block in this program.
 

[ClassName: 'Win32ASMFileMappingClass' 0
 AppName: 'Win32 ASM File Mapping Example' 0
 MenuName: 'FirstMenu' 0
 FilterString: 'All Files' 0, '*.*' 0
               'Text Files' 0, '*.txt' 0 0]

[hwnd: 0  MapFileHandle: 0  ReadFileHandle: 0  WriteFileHandle: 0   hMenu: 0     pMemory: 0
 SizeWritten: 0  FileHandle: 0  FileSize: 0  MenuHandle: 0]
 
_____________________________________________________________________________________
 

[IDM_EXIT 1  IDM_ABOUT 2  IDC_EDIT 3000  IDC_BUTTON 3001  IDC_EXIT 3002  DialogID 1000]
 

[MenuID  2000    M00_File  2001   M00_Open  2002   M00_Save_as  2003   M00_Exit  2004]

______________________________________________________________________________________

; Window Class Structure:

[DLGWINDOWEXTRA 30]

[WindowClassEx: wc_Size: len        wc_style: &CS_HREDRAW+&CS_VREDRAW   WndProc: MainWindowProc
                wc_ClsExtra: 0      wc_WndExtra: DLGWINDOWEXTRA       hInstance: 0
                wc_hIcon: 0         wc_hCursor: 0                     wc_hbrBackground: 6
                wc_MenuName: MenuID wc_ClassName: ClassName           wc_hIconSm: 0]
 

[UserFileFilter: 0 # 50] [FullChoosenFile: 0 #50] [ChoosenFile: 0 #20]
 

[OFN_FLAGS &OFN_FILEMUSTEXIST+&OFN_PATHMUSTEXIST+&OFN_LONGNAMES+&OFN_EXPLORER+&OFN_HIDEREADONLY]
 

[OpenFileNameStructure:  len  hwndFileOwner: 0  OF_hInstance: 0  FilterString
                         userFileFilter  200  &NULL  FullChoosenFile  200  ChoosenFile
                         200  &NULL  OpenFileTitle  OFN_FLAGS
                         nFileOffsetinChoosenFile: W§ 0  nFileExtensioninChoosenFile: 0
                         DefaultExtension: D§ &NULL
                         HookCustomData:  &NULL  HookProcPtr: &NULL  HookTemplateName: 0]
 
[OpenFileTitle:        '-=Our First Open File Dialog Box=-: Choose the file to open' 0]

[EditClass: 'edit'0] [EditID 1]
 
 

[FirstMessage: 0 #7]
____________________________________________________________________________________________
____________________________________________________________________________________________

Main:

    call 'Kernel32.GetModuleHandleA' 0 | mov D§hInstance eax, D§OF_hinstance eax

    call 'User32.LoadIconA' &NULL &IDI_APPLICATION | mov D§wc_hIcon eax, D§wc_hIconSm eax

    call 'User32.LoadCursorA' 0 &IDC_ARROW | mov D§wc_hCursor eax
    call 'User32.RegisterClassExA'  WindowClassEx
    call 'User32.CreateWindowExA' &WS_EX_CLIENTEDGE  ClassName  AppName,
                                 &WS_OVERLAPPEDWINDOW &CW_USEDEFAULT &CW_USEDEFAULT 300 200 ,
                                 &NULL &NULL D§hInstance 0
      mov D§hwnd eax, D§hwndFileOwner eax
 
    call 'User32.ShowWindow'  D§hwnd &SW_SHOWNORMAL | call 'User32.UpdateWindow'  D§hwnd
 
    jmp L1>
 
L0:     call 'User32.TranslateMessage'  Firstmessage
        call 'User32.DispatchMessageA'  Firstmessage

L1: call 'User32.GetMessageA' FirstMessage 0 0 0

    cmp eax 0 | ja L0<

    call 'Kernel32.ExitProcess' 0
 
_______________________________________________________________________________________

[AddString | mov esi #1 | While B§esi ne 0 | movsb | End_While]
[NextLine | mov al CR | stosb | mov al LF | stosb]

[EditHandle: 0]
 

Proc MainWindowProc:
    Arguments @Adressee, @Message, @wParam, @lParam
 
    pushad

    .If D@Message = &WM_DESTROY
        On D§MapFileHandle ne 0, call CloseMapFile
        call 'User32.PostQuitMessage' &NULL
 
    .Else_If D@Message = &WM_COMMAND
        mov eax D@wParam
        ..If D@lParam = 0
            ...If ax = M00_Open
                call 'Comdlg32.GetOpenFileNameA' OpenFileNameStructure
                    On eax = &TRUE, call LoadFile
                    call 'User32.SetFocus' EditHandle
 
            ...Else_If ax = M00_Save_as
                call 'Comdlg32.GetSaveFileNameA' OpenFileNameStructure
                On eax = &TRUE, call WriteFile
                call 'User32.SetFocus' EditHandle
            ...Else
                call 'User32.DestroyWindow' D§hWnd
            ...End_If
        ..End_If

    .Else_If D@Message = &WM_CREATE
        call 'User32.GetMenu' D§hWnd | mov D§MenuHandle eax
 

    .Else_If D@Message = &WM_SIZE
        mov eax D@lParam,  edx eax | shr edx 16 | and eax 0FFFF
        call 'User32.MoveWindow' D§EditHandle 0 0 eax edx &TRUE

    .Else
        popad
        call 'User32.DefWindowProcA' D@Adressee D@Message D@wParam D@lParam
        Exit

    .End_If
 
     popad | mov eax &FALSE
EndP
 
_______________________________________________________________________________________

LoadFile:

    call 'KERNEL32.CreateFileA' FullChoosenFile  &GENERIC_READ  &NULL,
                               &NULL  &OPEN_EXISTING   &FILE_ATTRIBUTE_ARCHIVE  &NULL
       mov D§ReadFileHandle eax
 
    call 'KERNEL32.CreateFileMappingA' D§ReadFileHandle &NULL &PAGE_READONLY 0 0 &NULL
    mov D§MapFileHandle eax
 
    mov eax FullChoosenFile | movzx edx W§nFileOffsetinChoosenFile | add eax edx
    call 'User32.SetWindowTextA' D§hWnd eax
 
    call 'User32.EnableMenuItem' D§MenuHandle M00_Open &MF_GRAYED
    call 'User32.EnableMenuItem' D§MenuHandle M00_Save_as &MF_ENABLED

  ret
 
_______________________________________________________________________________________
 

WriteFile:
    call 'KERNEL32.CreateFileA' FullChoosenFile,
                               &GENERIC_READ+&GENERIC_WRITE,
                               &FILE_SHARE_READ+&FILE_SHARE_WRITE,
                               &NULL &CREATE_NEW  &FILE_ATTRIBUTE_ARCHIVE  &NULL
       mov D§WriteFileHandle eax

    call 'KERNEL32.MapViewOfFile' D§MapFileHandle &FILE_MAP_READ 0 0 0
      mov D§pMemory eax
 
    call 'KERNEL32.GetFileSize' D§ReadFileHandle &NULL
    mov D§SizeWritten 0
 
    call 'KERNEL32.WriteFile' D§WriteFileHandle D§pMemory eax SizeWritten &NULL
 
    call 'KERNEL32.UnmapViewOfFile' D§pMemory
    call CloseMapFile
 
    call 'KERNEL32.CloseHandle' D§WriteFileHandle
 
    call 'User32.SetWindowTextA' D§hWnd AppName
    call 'User32.EnableMenuItem' D§MenuHandle M00_Open &MF_ENABLED
    call 'User32.EnableMenuItem' D§MenuHandle M00_Save_as &MF_GRAYED

  ret

______________________________________________________________________________________
 

CloseMapFile:
    call 'KERNEL32.CloseHandle' D§MapFileHandle | mov D§MapFileHandle 0
 
    call 'KERNEL32.CloseHandle' D§ReadFileHandle
  ret
 

Analysis:

 
    call 'KERNEL32.CreateFileA' FullChoosenFile  &GENERIC_READ  &NULL,
                               &NULL  &OPEN_EXISTING   &FILE_ATTRIBUTE_ARCHIVE  &NULL
       mov D§ReadFileHandle eax

When the user selects a file in the open file dialog, we call CreateFile to open it. Note that we specify GENERIC_READ to open this file for read-only access and dwShareMode is zero because we don't want some other process to modify the file during our operation.

 
   call 'KERNEL32.CreateFileMappingA' D§ReadFileHandle &NULL &PAGE_READONLY 0 0 &NULL
    mov D§MapFileHandle eax

Then we call CreateFileMapping to create a memory mapped file from the opened file. CreateFileMapping has the following syntax:

CreateFileMapping proto hFile:DWORD,\
                                         lpFileMappingAttributes:DWORD,\
                                         flProtect:DWORD,\
                                         dwMaximumSizeHigh:DWORD,\
                                         dwMaximumSizeLow:DWORD,\
                                         lpName:DWORD

You should know first that CreateFileMapping doesn't have to map the whole file to memory. You can use this function to map only a part of the actual file to memory. You specify the size of the memory mapped file in dwMaximumSizeHigh and dwMaximumSizeLow params. If you specify the size that 's larger than the actual file, the actual file will be expanded to the new size. If you want the memory mapped file to be the same size as the actual file, put zeroes in both params.
You can use NULL in lpFileMappingAttributes parameter to have Windows creates a memory mapped file with default security attributes.
flProtect defines the protection desired for the memory mapped file. In our example, we use PAGE_READONLY to allow only read operation on the memory mapped file. Note that this attribute must not contradict the attribute used in CreateFile else CreateFileMapping will fail.
lpName points to the name of the memory mapped file. If you want to share this file with other process, you must provide it a name. But in our example, our process is the only one that uses this file so we ignore this parameter.
 

    mov eax FullChoosenFile | movzx edx W§nFileOffsetinChoosenFile | add eax edx
    call 'User32.SetWindowTextA' D§hWnd eax

If CreateFileMapping is successful, we change the window caption to the name of the opened file. The filename with full path is stored in buffer, we want to display only the filename in the caption so we must add the value of nFileOffset member of the OPENFILENAME structure to the address of buffer.

 
    call 'User32.EnableMenuItem' D§MenuHandle M00_Open &MF_ENABLED
    call 'User32.EnableMenuItem' D§MenuHandle M00_Save_as &MF_GRAYED

As a precaution, we don't want the user to open multiple files at once, so we gray out the Open menu item and enable the Save menu item. EnableMenuItem is used to change the attribute of menu item.
After this, we wait for the user to select File/Save as menu item or close our program. If the user chooses to close our program, we must close the memory mapped file and the actual file like the code below:
 

    .If D@Message = &WM_DESTROY
        On D§MapFileHandle ne 0, call CloseMapFile
        call 'User32.PostQuitMessage' &NULL

In the above code snippet, when the window procedure receives the WM_DESTROY message, it checks the value of hMapFile first whether it is zero or not. If it's not zero, it calls CloseMapFile function which contains the following code:
 

CloseMapFile:
    call 'KERNEL32.CloseHandle' D§MapFileHandle | mov D§MapFileHandle 0
 
    call 'KERNEL32.CloseHandle' D§ReadFileHandle
  ret

CloseMapFile closes the memory mapped file and the actual file so that there 'll be no resource leakage when our program exits to Windows.
If the user chooses to save that data to another file, the program presents him with a save as dialog box. After he types in the name of the new file, the file is created by CreateFile function.
 

    call 'KERNEL32.MapViewOfFile' D§MapFileHandle &FILE_MAP_READ 0 0 0
      mov D§pMemory eax

Immediately after the output file is created, we call MapViewOfFile to map the desired portion of the memory mapped file into memory. This function has the following syntax:

MapViewOfFile proto hFileMappingObject:DWORD,\
                                   dwDesiredAccess:DWORD,\
                                   dwFileOffsetHigh:DWORD,\
                                   dwFileOffsetLow:DWORD,\
                                   dwNumberOfBytesToMap:DWORD

dwDesiredAccess specifies what operation we want to do to the file. In our example, we want to read the data only so we use FILE_MAP_READ.
dwFileOffsetHigh and dwFileOffsetLowspecify the starting file offset of the file portion that you want to map into memory. In our case, we want to read in the whole file so we start mapping from offset 0 onwards.
dwNumberOfBytesToMap specifies the number of bytes to map into memory. If you want to map the whole file (specified by CreateFileMapping), pass 0 to MapViewOfFile.
After calling MapViewOfFile, the desired portion is loaded into memory. You'll be given the pointer to the memory block that contains the data from the file.

    call 'KERNEL32.GetFileSize' D§ReadFileHandle &NULL
    mov D§SizeWritten 0

Find out how large the file is. The file size is returned in eax. If the file is larger than 4 GB,  the high DWORD of the file size is stored in FileSizeHighWord. Since we don't expect to handle such large file, we can ignore it.

    call 'KERNEL32.WriteFile' D§WriteFileHandle D§pMemory eax SizeWritten &NULL

Write the data that is mapped into memory into the output file.

    call 'KERNEL32.UnmapViewOfFile' D§pMemory

When we're through with the input file, unmap it from memory.

    call 'KERNEL32.UnmapViewOfFile' D§pMemory
    call CloseMapFile
 
    call 'KERNEL32.CloseHandle' D§WriteFileHandle

And close all the files.

    call 'User32.SetWindowTextA' D§hWnd AppName

Restore the original caption text.

    call 'User32.EnableMenuItem' D§MenuHandle M00_Open &MF_ENABLED
    call 'User32.EnableMenuItem' D§MenuHandle M00_Save_as &MF_GRAYED

Enable the Open menu item and gray out the Save As menu item.


[Iczelion's Win32 Assembly HomePage]