StackOverflow2013

Note that there are some explanatory texts on larger screens.

plurals

POIs it possible to cache mapped regions returned from MapViewOfFile?
text
Body
copied!<p>Good afternoon, It is a well known fact that when dealing with large files that cannot be mapped to one view in Win32, create code that carefully maps and unmaps file regions as they are needed. The pastebin url is:</p> <p>I created and tested a cMemoryMappedFile class that deals with large files that cannot be mapped to one view in Win32. I tested the class and found that while it functions OK, it takes a long time(i.e 3 seconds) for random access. This is because the class has to unmap and map a file region for every random access. I was wondering if it was possible to cache the mapped regions returned from MapViewFile to speed up random access.</p> <p>Yesterday, I noticed that UnMapViewOfFile invalidates a previously mapped region returned from MapViewOfFile. Does anyone have ideas about how to speed up random access through caching or other methods?</p> <p>Currently the viewport is 128KB. I believe that if I enlarge the viewport it will reduce the number of calls to UnMapViewOfFile and MapViewOfFile. However, I was wondering if could use other methods. Please look at the method,<br> char* cMemoryMappedFile::GetPointer(int , bool) to see how the viewport is shifted with the file mapping. Thank you.</p> <p>The pastebin url for the class is<br> > . I am adding the source code here in case no one can access the url.</p> <pre><code>// cMemoryMappedFile.Cpp #include "cException.h" #include "cMemoryMappedFile.h" #define BUFFER_SIZE 10 #define MEM_BLOCK_SIZE 65536 * 2 /** \class cMemoryMappedFile \brief Encapsulation of the Windows Memory Management API. The cMemoryMapped class makes some memory mapping operations easier. */ /** \brief Constructor for cMemoryMappedFile object. \param FileSize Size of file. \param OpenMode File open mode \param AccessModes File access mode \param ShareMode File sharing mode \param Flags File attributes and flags \param ShareMode File sharing mode \param Flags File attributes and flags \param Security Security Attributes \param Template Extended attributes tp apply to a newly created file */ cMemoryMappedFile::cMemoryMappedFile(long FileSize_, OpenModes OpenMode_,AccessModes AccessMode_, ShareModes ShareMode_,long Flags_,void *Security_,FILEHANDLE Template_) { FileSize = FileSize_; char buffer[BUFFER_SIZE]; DWORD dwRetVal = 0; UINT uRetVal = 0; DWORD dwPtr = 0; BOOL isSetEndOfFile = FALSE; LARGE_INTEGER Distance_; DWORD ErrorCode = 0; char lpTempPathBuffer[MAX_PATH]; PreviousNCopy = 0; PreviousN = 0; // Gets the temp path env string (no guarantee it's a valid path). dwRetVal = GetTempPath(MAX_PATH, // length of the buffer lpTempPathBuffer); // buffer for path if (dwRetVal > MAX_PATH || (dwRetVal == 0)) { throw cException(ERR_MEMORYMAPPING,""); } // Generates a temporary file name. uRetVal = GetTempFileName(lpTempPathBuffer, // directory for tmp files TEXT("DEMO"), // temp file name prefix 0, // create unique name TempFileName); // buffer for name if (uRetVal == 0) { throw cException(ERR_MEMORYMAPPING,lpTempPathBuffer); } // Creates the new file hFile = CreateFile((LPTSTR) TempFileName, // file name AccessMode_, // open for write 0, // do not share (SECURITY_ATTRIBUTES *) Security_, // default security OpenMode_, // CREATE_ALWAYS, Flags_,// normal file Template_); // no template if (hFile == INVALID_HANDLE_VALUE) { throw cException(ERR_MEMORYMAPPING,TempFileName); } Distance_.LowPart = (ULONG)FileSize_; Distance_.HighPart = 0; // (ULONG)(FileSize_ >> 32); dwPtr = ::SetFilePointer(hFile,Distance_.LowPart, &(Distance_.HighPart), FileBegin); if (dwPtr == INVALID_SET_FILE_POINTER){ throw cException(ERR_MEMORYMAPPING,TempFileName); } isSetEndOfFile = SetEndOfFile(hFile); if (!isSetEndOfFile){ ErrorCode = GetLastError(); throw cException(ERR_MEMORYMAPPING,TempFileName); } hMapping=::CreateFileMapping(hFile,(SECURITY_ATTRIBUTES *)Security_,PAGE_READWRITE,0,0,0); if (hMapping==NULL) throw cException(ERR_MEMORYMAPPING,TempFileName); MapPtr = 0; adjustedptr = 0; prevadjustedptr = adjustedptr; FilePath=new char[strlen(TempFileName)+1]; strcpy(FilePath,TempFileName); } char * cMemoryMappedFile::GetPointer(int n, bool Caching){ unsigned int baseoff; if( n < MEM_BLOCK_SIZE / 2) { baseoff = 0; } else { baseoff = ((n + MEM_BLOCK_SIZE / 4) & (~(MEM_BLOCK_SIZE / 2 - 1))) - MEM_BLOCK_SIZE / 2; } // the correct memory mapped view is already mapped in if (adjustedptr != 0 && mappedoffset == baseoff && Caching) return adjustedptr; else if (Caching) { /* retrieve adjustedptr from cache */ } // get a new memory mapped viewport else{ if (MapPtr){ UnmapViewOfFile(MapPtr); PreviousNCopy = PreviousN; prevadjustedptr = adjustedptr; } PreviousN = n; mappedlength = min(FileSize - baseoff, MEM_BLOCK_SIZE); // MapViewOfFile should be aligned to 64K boundary MapPtr = (char*)::MapViewOfFile( hMapping, FILE_MAP_WRITE | FILE_MAP_READ, 0, baseoff, mappedlength); mappedoffset = baseoff; adjustedptr = MapPtr - mappedoffset; printf("Value: %u n: %u\n",adjustedptr[n],n); /* cache PreviousNCopy,PreviousN, prevadjustedptr[PreviousNCopy] */ } return adjustedptr; } </code></pre>

Querying!

Guidance

An individual column

Larger individual text columns get their own page to allow for proper reading.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload