Guest Geoff C Posted September 21, 2007 Posted September 21, 2007 NTFS not maintaining "last access time" correctly Operating system: Windows XP Pro (Version 5.1.2600, Service Pack 2, Build 2600). Expected behaviour: According to the system documentation (eg: Fsutil behavior - disablelastaccess parameter), NTFS maintains "last access time" (LAT) values of files as follows: <quote> Each file and folder on an NTFS volume contains an attribute called Last Access Time. This attribute defines when the file or folder was last accessed, such as when a user lists folders, adds files to a folder, reads a file, or makes changes to a file. The most up-to-date Last Access Time is stored in memory and is eventually written to the disk in two different locations. [1] One is within the file's attribute, which is part of its MFT record. The second is in the index of the directory that contains the file. The Last Access Time on disk is not always current. This lag occurs because NTFS delays writing the Last Access Time to disk when users or programs perform read-only operations on a file or folder, such as listing the folder's contents or reading (but not changing) a file in the folder. If the Last Access Time is kept current on disk for read operations, all read operations become write operations, which impacts NTFS performance. Note that file-based queries of Last Access Time are accurate even if all on-disk values are not current. NTFS returns the correct value on queries because the accurate value is stored in memory. [2] NTFS typically updates a file's attribute on disk if the current Last Access Time in memory differs by more than an hour from the Last Access Time stored on disk, or when all in-memory references to that file are gone, whichever is more recent. For example, if a file's current Last Access Time is 1:00 P.M., and you read the file at 1:30 P.M., NTFS does not update the Last Access Time. If you read the file again at 2:00 P.M., NTFS updates the Last Access Time in the file's attribute to reflect 2:00 P.M. because the file's attribute shows 1:00 P.M. and the in-memory Last Access Time shows 2:00 P.M. NTFS updates the index of the directory that contains the file when NTFS updates the file's Last Access Time and detects that the Last Access Time for the file differs by more than an hour from the Last Access Time stored in the directory's index. This update typically occurs after a program closes the handle used to access a file within the directory. If the user holds the handle open for an extended time, a lag occurs before the change appears in the index entry of the directory. Note that one hour is the maximum time that NTFS defers updating the Last Access Time on disk. [3] If NTFS updates other file attributes such as Last Modify Time, and a Last Access Time update is pending, NTFS updates the Last Access Time along with the other updates without additional performance impact. </quote> Actual behaviour: This is not the way NTFS is behaving. From the points annotated in the quote above: [1] If this in-memory, most up-to-date LAT does indeed exist, it is never written out to disc. [2] File-based queries return the on-disc value: or certainly not the most up-to-date LAT at least. [3] The one hour period is refractory to further file accesses. The update is not deferred: rather, it is never performed. Further accesses during the hour are merely ignored. Illustrative example: A file is accessed at 1:00 pm by opening it with Notepad, then closing the program without modifying the file. This will cause the LAT of the file to be set to 1:00 pm. At 1:30pm the process is repeated. The LAT is then queried by examining the properties of the file. [2] The reported time is 1:00 pm, the stale on-disc LAT, not the most up-to-date in-memory value, which should be 1:30 pm (or indeed, the present time, since querying the properties constitutes accessing the file, in Windows). The file's LAT can be checked in command line (dir /ta) at 3:00 pm (or whenever) - it is never updated by NTFS and certainly not within one hour [3]. It makes no difference if the session is logged off, or Windows shutdown - the 1:30 pm access is never recorded to disc. In fact, NTFS just ignores any subsequent file access occurring within one hour of the initial access. If the file is accessed again at 2:00 pm, the LAT will be updated to reflect this time. However, the file can be accessed repeatedly within the hour from 1:00 pm to 2:00 pm without this ever being recorded permanently by NTFS. As such, LAT values may be incorrect by up to one hour, not just temporarily until a deferred update, but permanently. The same deficiencies are found for other actions considered by Windows to constitute a file access, such as: - querying file properties via the context menu - hovering over a file icon to elicit an information ‘tool tip’ - selecting a file icon by single-click I have checked this on another machine with the same operating system, so I am confident this is not just a manifestation of my local system. Presumably, it is possible this also affects other versions of Windows making use of the NTFS modules.
Guest Poprivet Posted September 21, 2007 Posted September 21, 2007 Re: NTFS not maintaining "last access time" correctly As useless as LAT is, I'm real curious why it matters? It's just one of those thaings that's only useful to MS during their development, IMO. If you're trying to determine user access, there are much better and accurate ways. Pop` Geoff C wrote: > NTFS not maintaining "last access time" correctly > > Operating system: Windows XP Pro (Version 5.1.2600, Service Pack 2, > Build 2600). > > Expected behaviour: > According to the system documentation (eg: Fsutil behavior - > disablelastaccess parameter), NTFS maintains "last access time" (LAT) > values of files as follows: > > <quote> > Each file and folder on an NTFS volume contains an attribute called > Last Access Time. This attribute defines when the file or folder was > last accessed, such as when a user lists folders, adds files to a > folder, reads a file, or makes changes to a file. The most up-to-date > Last Access Time is stored in memory and is eventually written to the > disk in two different locations. [1] One is within the file's > attribute, which is part of its MFT record. The second is in the > index of the directory that contains the file. The Last Access Time > on disk is not always current. This lag occurs because NTFS delays > writing the Last Access Time to disk when users or programs perform > read-only operations on a file or folder, such as listing the > folder's contents or reading (but not changing) a file in the folder. > If the Last Access Time is kept current on disk for read operations, > all read operations become write operations, which impacts NTFS > performance. > Note that file-based queries of Last Access Time are accurate even if > all on-disk values are not current. NTFS returns the correct value on > queries because the accurate value is stored in memory. [2] > NTFS typically updates a file's attribute on disk if the current Last > Access Time in memory differs by more than an hour from the Last > Access Time stored on disk, or when all in-memory references to that > file are gone, whichever is more recent. For example, if a file's > current Last Access Time is 1:00 P.M., and you read the file at 1:30 > P.M., NTFS does not update the Last Access Time. If you read the file > again at 2:00 P.M., NTFS updates the Last Access Time in the file's > attribute to reflect 2:00 P.M. because the file's attribute shows > 1:00 P.M. and the in-memory Last Access Time shows 2:00 P.M. NTFS > updates the index of the directory that contains the file when NTFS > updates the file's Last Access Time and detects that the Last Access > Time for the file differs by more than an hour from the Last Access > Time stored in the directory's index. This update typically occurs > after a program closes the handle used to access a file within the > directory. If the user holds the handle open for an extended time, a > lag occurs before the change appears in the index entry of the > directory. > Note that one hour is the maximum time that NTFS defers updating the > Last Access Time on disk. [3] If NTFS updates other file > attributes such as Last Modify Time, and a Last Access Time update is > pending, NTFS updates the Last Access Time along with the other > updates without additional performance impact. > </quote> > > Actual behaviour: > This is not the way NTFS is behaving. From the points annotated in > the quote above: > [1] If this in-memory, most up-to-date LAT does indeed exist, it is > never written out to disc. > [2] File-based queries return the on-disc value: or certainly not the > most up-to-date LAT at least. > [3] The one hour period is refractory to further file accesses. The > update is not deferred: rather, it is never performed. Further > accesses during the hour are merely ignored. > > Illustrative example: > A file is accessed at 1:00 pm by opening it with Notepad, then > closing the program without modifying the file. This will cause the > LAT of the file to be set to 1:00 pm. At 1:30pm the process is > repeated. The LAT is then queried by examining the properties of the > file. [2] The reported time is 1:00 pm, the stale on-disc LAT, not > the most up-to-date in-memory value, which should be 1:30 pm (or > indeed, the present time, since querying the properties constitutes > accessing the file, in Windows). > > The file's LAT can be checked in command line (dir /ta) at 3:00 pm (or > whenever) - it is never updated by NTFS and certainly not within one > hour [3]. It makes no difference if the session is logged off, or > Windows shutdown - the 1:30 pm access is never recorded to disc. > > In fact, NTFS just ignores any subsequent file access occurring > within one hour of the initial access. If the file is accessed again > at 2:00 pm, the LAT will be updated to reflect this time. However, > the file can be accessed repeatedly within the hour from 1:00 pm to > 2:00 pm without this ever being recorded permanently by NTFS. As > such, LAT values may be incorrect by up to one hour, not just > temporarily until a deferred update, but permanently. > > The same deficiencies are found for other actions considered by > Windows to constitute a file access, such as: > - querying file properties via the context menu > - hovering over a file icon to elicit an information 'tool tip' > - selecting a file icon by single-click > > I have checked this on another machine with the same operating > system, so I am confident this is not just a manifestation of my > local system. Presumably, it is possible this also affects other > versions of Windows making use of the NTFS modules.
Guest Geoff C Posted September 28, 2007 Posted September 28, 2007 Re: NTFS not maintaining "last access time" correctly The reason why this matters is because people may think there is a problem with their system if it is not behaving the way it is supposed to. At the very least, the system documentation should be altered to reflect the fact that the access times aren't maintained correctly. System documentation is either pointless or harmful if it is wrong. This is exactly what happened to me. The reason I found this issue with NTFS is because I have had other problems with my system recently, so I was probably being hyper-vigiilant about anything behaving strangely. It then cost me heaps of time to get to the bottom of why the last access times weren't being handled correctly. I assumed it was symptomatic of further corruption arising from the earlier problems with my system. Eventually, I found it wasn't a problem with my system at all, but rather it is a bug in NTFS. This is why I reported the issue here, because someone in future may have the same confusion. I tried reporting it to MS, but just got the usual duck-shove treatment - I doubt they even read my email. Service is great when you're at the buying stage, but after that, go tell someone who cares... "Poprivet" wrote: > As useless as LAT is, I'm real curious why it matters? It's just one of > those thaings that's only useful to MS during their development, IMO. If > you're trying to determine user access, there are much better and accurate > ways. > > Pop` > > > > > Geoff C wrote: > > NTFS not maintaining "last access time" correctly > > > > Operating system: Windows XP Pro (Version 5.1.2600, Service Pack 2, > > Build 2600). > > > > Expected behaviour: > > According to the system documentation (eg: Fsutil behavior - > > disablelastaccess parameter), NTFS maintains "last access time" (LAT) > > values of files as follows: > > > > <quote> > > Each file and folder on an NTFS volume contains an attribute called > > Last Access Time. This attribute defines when the file or folder was > > last accessed, such as when a user lists folders, adds files to a > > folder, reads a file, or makes changes to a file. The most up-to-date > > Last Access Time is stored in memory and is eventually written to the > > disk in two different locations. [1] One is within the file's > > attribute, which is part of its MFT record. The second is in the > > index of the directory that contains the file. The Last Access Time > > on disk is not always current. This lag occurs because NTFS delays > > writing the Last Access Time to disk when users or programs perform > > read-only operations on a file or folder, such as listing the > > folder's contents or reading (but not changing) a file in the folder. > > If the Last Access Time is kept current on disk for read operations, > > all read operations become write operations, which impacts NTFS > > performance. > > Note that file-based queries of Last Access Time are accurate even if > > all on-disk values are not current. NTFS returns the correct value on > > queries because the accurate value is stored in memory. [2] > > NTFS typically updates a file's attribute on disk if the current Last > > Access Time in memory differs by more than an hour from the Last > > Access Time stored on disk, or when all in-memory references to that > > file are gone, whichever is more recent. For example, if a file's > > current Last Access Time is 1:00 P.M., and you read the file at 1:30 > > P.M., NTFS does not update the Last Access Time. If you read the file > > again at 2:00 P.M., NTFS updates the Last Access Time in the file's > > attribute to reflect 2:00 P.M. because the file's attribute shows > > 1:00 P.M. and the in-memory Last Access Time shows 2:00 P.M. NTFS > > updates the index of the directory that contains the file when NTFS > > updates the file's Last Access Time and detects that the Last Access > > Time for the file differs by more than an hour from the Last Access > > Time stored in the directory's index. This update typically occurs > > after a program closes the handle used to access a file within the > > directory. If the user holds the handle open for an extended time, a > > lag occurs before the change appears in the index entry of the > > directory. > > Note that one hour is the maximum time that NTFS defers updating the > > Last Access Time on disk. [3] If NTFS updates other file > > attributes such as Last Modify Time, and a Last Access Time update is > > pending, NTFS updates the Last Access Time along with the other > > updates without additional performance impact. > > </quote> > > > > Actual behaviour: > > This is not the way NTFS is behaving. From the points annotated in > > the quote above: > > [1] If this in-memory, most up-to-date LAT does indeed exist, it is > > never written out to disc. > > [2] File-based queries return the on-disc value: or certainly not the > > most up-to-date LAT at least. > > [3] The one hour period is refractory to further file accesses. The > > update is not deferred: rather, it is never performed. Further > > accesses during the hour are merely ignored. > > > > Illustrative example: > > A file is accessed at 1:00 pm by opening it with Notepad, then > > closing the program without modifying the file. This will cause the > > LAT of the file to be set to 1:00 pm. At 1:30pm the process is > > repeated. The LAT is then queried by examining the properties of the > > file. [2] The reported time is 1:00 pm, the stale on-disc LAT, not > > the most up-to-date in-memory value, which should be 1:30 pm (or > > indeed, the present time, since querying the properties constitutes > > accessing the file, in Windows). > > > > The file's LAT can be checked in command line (dir /ta) at 3:00 pm (or > > whenever) - it is never updated by NTFS and certainly not within one > > hour [3]. It makes no difference if the session is logged off, or > > Windows shutdown - the 1:30 pm access is never recorded to disc. > > > > In fact, NTFS just ignores any subsequent file access occurring > > within one hour of the initial access. If the file is accessed again > > at 2:00 pm, the LAT will be updated to reflect this time. However, > > the file can be accessed repeatedly within the hour from 1:00 pm to > > 2:00 pm without this ever being recorded permanently by NTFS. As > > such, LAT values may be incorrect by up to one hour, not just > > temporarily until a deferred update, but permanently. > > > > The same deficiencies are found for other actions considered by > > Windows to constitute a file access, such as: > > - querying file properties via the context menu > > - hovering over a file icon to elicit an information 'tool tip' > > - selecting a file icon by single-click > > > > I have checked this on another machine with the same operating > > system, so I am confident this is not just a manifestation of my > > local system. Presumably, it is possible this also affects other > > versions of Windows making use of the NTFS modules. > > > >
Recommended Posts