Monitor Huge Directories for Changes

Just a few quick tips if you want to detect new, changed or deleted files in *huge* folders.

The most important setting (performance wise) in WatchDirectory is the Monitoring Method. You can select between several methods, but only the "Fast" method is independent of the size of the monitored directory. The other three methods work by scanning the directory for changes - large directories take more time and memory to process.
The "Fast" monitoring method doesn't need to scan the directory - Windows will inform the task of the exact change that happened.

If you can not use the "Fast" method, make sure to exclude all subfolders you do not want to monitor. Also filter out unneeded files and/or directories.
If you have the choice between "exclude sub-folders" and "filter", the optimum way (performance) is to exclude subfolders. An excluded subfolder is never scanned for changes. The filter events settings are applied at a much later stage - after the directories are scanned.

Even if you do use the "Fast" monitoring method, in some cases WatchDirectory will actually "scan" the directory. This happens when you

• Select Fire FILENEW/DIRNEW events for all files at startup or When starting up, check for missed events on the Other Settings window.
• Select Trigger for all files and subdirectories below the renamed directory on the Events to detect window.

I hope this post helps you understand the effect of the options you select in WatchDirectory.