File Viking - Status

I mentioned this new program in an earlier post. As always, progress is slower than originally planned - this says more about our planning- than our programming-capabilities, by the way ;-)

Beta Testers

We can still use some beta testers for this new program. Beta testers receive a free copy when it is released, obviously. Interested? We expect to have the first beta available somewhere next month.

If you want to help, send an email to gert@watchdirectory.net, use the subject "SKUNKWORKS - Beta".

The Truth About Amsterdam

The capitol of The Netherlands, Amsterdam, received some bad press by TV host Bill O’Reilly from Fox News. Two inhabitants of Amsterdam created a website called The Truth About Amsterdam where they defend Amsterdam. Below are 2 video's with their answer to Bill. More videos are available on their website.

The truth about Amsterdam, our response to Bill

Amsterdam versus O’Reilly 2

WatchFTP 2.2 beta 2

We have a new beta for WatchFTP: 2.2 beta 2.

New in this beta:

• Previously, files bigger than 4GB would not download properly.
• French language support (helpfile is not translated yet).
  We are very interested to here about the quality of this translation. Any remarks you have, please send an email to eric@watchftp.com
• Separate installers for each supported language.
  Previously, the user interface language was detected by the operating system. There was no way to use the English interface on a German language operating system. This new version will bypass the automatic detection and always use the language depending on the download you have installed.

Download links can be found in this forum post.

Many tasks, little time

The last WatchDirectory update introduced several features that help people who run a lot of WatchDirectory tasks. This post describes them in more detail and introduces a little undocumented feature.

Find Tasks

Version 4.5.9 has a new option to Find Tasks. Use the menu "View -> Find" or the keyboard shortcut CTRL+F to invoke it. Enter any search term and the WD Control Center will search all visible columns for the entered text. All tasks that have a matching text will be selected and optionally scrolled to top.

There is also an option to "Clear previous results". If you deselect it you can do multiple "finds" to select all the tasks you want.

When all desired tasks are selected you can Start them all at once by clicking the "Play" button. If one of the tasks selected is already running you will find the play-button is disabled (grayed). To enable the play-button you must either:

• De-select the running tasks:
  Ctrl+Click the selected running task to de-select it
• Force the Play-button to become active:
  Hold down CTRL and SHIFT while you click play. Only the not-running selected tasks will be started.

Group Tasks by Color

Also new: you can assign one of 6 colors to a task. Select one or more tasks and right-click them. Select the menu-option "Set Color" and assign a color to the selected task(s). After doing this for the first time, 2 things will happen:

• A new column will show
  This column contains a colored "dot" for the tasks. Clicking the column header will sort all tasks by their assigned color.
• A new color toolbar will show
  This toolbar allows you to quickly select all tasks having the same color. You can disable this new toolbar using the "View ->Toolbars" menu option.

So now you can, for example, assign "green" to all tasks related to "Incoming Customer Orders" and assign "red" to all tasks related to "Fulfilled Orders".

Click the red dot on the new toolbar to select all "Fulfilled Orders" tasks and start or stop them all at once.

Undocumented - named colors

Maybe "red" is not so obvious... You may be wondering, was it green or red I used for "Fulfilled Orders"?? When you hover your mouse over the colors toolbar, the Control Center will read the description to show from the Windows Registry. If not found there, it will use the default (non descriptive) text for this color to show.

If you want easier, intuitive descriptions for these colors, download this file: ColorDescriptions.txt. (you must right-click, "save as" this link, otherwise it may show as rubbish in your browser). Save it to your desktop and open it with notepad (or another plain-text editor). Inside the file you find several lines like:

"1"="Statusbar message for Red\\npopup for red"

(followed by "2"=, "3"=..."6"=, one for each color). Change the text anyway you like, but please make sure it keeps this same format (especially the "\\n" part is important). For example, change this line to

"1"="Select all Fulfilled Orders\\nFulfilled Orders"

The text before "\\n" is showed on the statusbar. The text after "\\n" is shown as a little yellow notification when you hover your mouse over this color.

After you made the desired changes to ColorDescriptions.txt, rename it to ColorDescriptions.reg (ignore warnings from Windows). Now double-click the ColorDescriptions.reg file to import these settings into the Windows Registry. When you restart the WatchDirectory Control Center, you will see your new descriptions.

Just a reminder

If you have bought a version 4.x license from us, you can upgrade this license free of charge. Just install the latest "evaluation version" on top of your current install so it will see your current settings, tasks and license info.

Download here or for the German lanuage version: here.

Q&A - Load Balancing Humans

Q&A

One of our customers, USA Transcription Services, uses WatchDirectory to automatically send work from clients to transcriptionists. Clients upload work to a directory monitored by WatchDirectory and WatchDirectory sends it to the transcriptionist assigned to the client.

Normally each client has a dedicated transcriptionist, but some clients send a lot of work to transcribe - more work than one employee can handle in a reasonable time. What is needed is to assign 2 transcriptionists to such a client, making sure no work is duplicated (done by both).

Lori, my contact at USA Transcription Services, asked if I knew a way to solve this.

Solution Outline

Instead of sending work directly from the directory where clients upload work, we first need to distribute these files into separate directories, assigned to different transciptionists. These directories can then be monitored by another WatchDirectory task to send the work to the individual employees.

Solution 1 - Sort Files

Use the Sort Files plugin to distribute the detected files to employee folders. Create multiple "sort rules", making sure they are all "final" (so a file is only copied to one employee folder).

This solution depends on the names of the files uploaded by the customers. If you can be sure these names are quite random, you can base the sort rules on, for example, the first letter of the filename. The first sort-rule would use a mask like

*\a*;*\b*;*\c*;*\d*;*\e*;*\f*;*\g*;*\h*

so it copies all files with names starting with a, b, c, d, e, f, g and h (ignoring case), and the second rule would just use * as the mask - matching all files not handled by the first rule.

One problem: the masks as entered for sort-rule one will also match the file C:\Directory\ZZZZZZZZ.TXT because of the mask "*\d*".
So, in this case, it is better to enter the file masks as *directory\d* (or just *ory\d*).

Another problem, it can be quite hard to guarantee all files uploaded by clients have random names. Maybe better to use masks based on the second character of those names:

*\?a*;*\?b*;*\?c*;*\?d*;*\?e*;*\?f*;*\?g*;*\?h*

This post is not intended to go into the linguistic analysis of filenames but I think the letter 'E' is quite common as the second letter in English filenames

Solution 2 - A Batch File that distributes randomly

This is the solution Lori is using now. If you don't need rocket science precision (exactly half of the files go to directory-1, the other half goes to directory-2), this will work fine. Especially if you are dealing with a large number of files to distribute.
This solution uses the Run a Batch File task to start a script that uses the environment variable %RANDOM% to determine the target directory (and thus the employee that transcribes the file).
Here is the script:

SET TARGET1=C:\Uploads\Employee1
SET TARGET2=C:\Uploads\Employee2
rem get a random number (0 - ~32000)
rem and get the remainder of divide by 2, so we have a number 0 or 1 as the result
SET /a REMAINDER=%RANDOM% %% 2
SET TARGET=%TARGET1%
IF %REMAINDER% EQU 1 SET TARGET=%TARGET2%
rem move the detected file to TARGET
MOVE "%WD_FILE%" "%TARGET%"

If a client needs 3 employees, the script can be changed to (changes highlighted):

SET TARGET1=C:\Uploads\Employee1
SET TARGET2=C:\Uploads\Employee2
SET TARGET3=C:\Uploads\Employee3
rem get a random number (0 - ~32000)
rem and get the remainder of divide by 3, so we have a number 0, 1 or 2 as the result
SET /a REMAINDER=%RANDOM% %% 3
SET TARGET=%TARGET1%
IF %REMAINDER% EQU 1 SET TARGET=%TARGET2%
IF %REMAINDER% EQU 2 SET TARGET=%TARGET3%
rem move the detected file to TARGET
MOVE "%WD_FILE%" "%TARGET%"

Solution 3 - No Randomness

Random can be a tricky concept... I will let Dilbert Explain. To guarantee an even distribution of files you need to count the files, please see this forum post.

PS: I found the Dilbert comic only on web.archive.org. It must be somewhere on the official dilbert.com site as well, but I could not find it. If you have a link to the original pic on dilbert.com, please let me know.

August Special - Free Agenda At Once license

If you buy 2 or more WatchDirectory or WatchFTP licenses in August 2009, you can get a free license of this Great Organizer and ToDo-list program. A $40 gift from us to you!

If you are already a customer and buy one extra license (remember to contact us first for a 25% discount), this offer for a free Agenda At Once license is also valid.

Simply forward your order-confirmation email to support@watchdirectory.net, please allow a few days to process your request.

About Agenda At Once

Agenda At Once PIM for Windows combines the best aspects of other personal organizer applications. It's a complete PIM system, integrating to-do management, scheduling and planning, contact and note functions. In addition, it is easy to use and responsive, while being feature-packed but not power-hungry. With its optimally-designed interface, Agenda At Once will rapidly put you in charge of your - or your team's - personal information. You'll be saving time and accomplishing so much more that you'll be wondering how you ever managed without it.

8 processors but still waiting

Our new build system has lots of ram and a multicore hyperthreaded cpu: 4*2 processors. It is just unbelievable how responsive this Vista-64 machine is.

We are currently making some changes to the build process of WatchFTP, creating separate installers for each supported language. Previously, we created just one installer with a multilingual executable (thank you Sisulizer for a great localization product). The multilingual approach worked fine most of the time, however when a German customer living in France wants to run the English version of WatchFTP on his Spanish-language operating system it was just not possible. The operating system would select the language to display, not much you could do about it.

Future WatchFTP versions will have the desired display language build-in. Sisulizer will create language specific (German, Spanish etc) copies of the original (English) executables. The display language will only depend on the installer you download from our websites.

Steps in our build process

1. Build a "release" version of WatchFTP
   (Microsoft Visual Studio 2008 - C++ by the way)
2. Run Sisulizer to create the multilingual executables
3. Protect the core components (2 executables)
4. Code-sign the EXE and DLLs
5. Run Inno Setup to create the installer
6. Code-sign the setup file
7. Test the setup
   When we find issues, fix them and go back to step 1.
8. When all is OK, a lot of other tasks like uploading to our website etc

The most time consuming step (several minutes) was step 3 (software protection). Not really a big deal, we don't release new versions that often. However, step 7 (test setup) can cause a lot of rebuilds (start at step 1) and the delays for software protection become very annoying.

It gets Worse!!

When we implement separate installers for the different languages we support, we will have to do step 3 (protect) for each language (currently German, Spanish and English. Soon we will have a French version as well).
This will make our already long build process 4 times longer - totally unacceptable.

Our great new computer doesn't help at all because all those steps need to be sequential. We can not code-sign (step 4) before the protection (step 3) is completed. Also, the protection phase is done with a single-threaded program, even if the computer has 1000 cpus, it will only use one of them.

How we solved it

Obviously, the protection step (nr 4) needs to be paralleled. We can launch the protector as a separate process for each of the languages. A first attempt (WRONG!!) in our build batch script:

....
ECHO ==STEP 2 Complete==>>C:\Temp\buildlog.txt
ECHO ==STEP 3 Starts - Software Protection==>>C:\Temp\buildlog.txt
Set LANGUAGE=ENGLISH
Start "Protecting %LANGUAGE%" "E:\Build\Protect.bat"
Set LANGUAGE=SPANISH
Start "Protecting %LANGUAGE%" "E:\Build\Protect.bat"
Set LANGUAGE=GERMAN
Start "Protecting %LANGUAGE%" "E:\Build\Protect.bat"
Set LANGUAGE=FRENCH
Start "Protecting %LANGUAGE%" "E:\Build\Protect.bat"
ECHO ==STEP 3 Complete==>>C:\Temp\buildlog.txt
ECHO ==STEP 4 Starts - Code Signing protected Executables ==>>C:\Temp\buildlog.txt

(note: the scripts shown in this post are very simplified without error checking etcetera).
The protect.bat file is called and based on the LANGUAGE variable it will protect a specific language version of WatchFTP. The beauty of the Start command is that it will launch a separate console (command prompt) to run the protect.bat script. In the example above 4 language versions are protected simultaneously, each using their own CPU. Obviously, because of disk usage, there is some slowdown - it won't complete 4 times as fast.

A small problem...

In case you didn't realize, the above script is sort of wrong: it continues running while the 4 "Protect.bat" files are not finished yet. So it gladly goes on to Step 4 (code signing), but there isn't anything to code-sign yet. We need to wait for the 4 protect.bat processes to complete before we can continue with Step 4.

And an easy fix

We have a small command prompt program called GdPUtil (Free Download). Please read the description of the -join parameter.

We added the following lines at the end of protect.bat so it signals to the main script it is ready:

....
"C:\Bin\GdPUtil.exe" -join protect%LANGUAGE% 1000
EXIT

And changed the original build script to

....
ECHO ==STEP 2 Complete==>>C:\Temp\buildlog.txt
ECHO ==STEP 3 Starts - Software Protection==>>C:\Temp\buildlog.txt
Set LANGUAGE=ENGLISH
Start "Protecting %LANGUAGE%" "E:\Build\Protect.bat"
Set LANGUAGE=SPANISH
Start "Protecting %LANGUAGE%" "E:\Build\Protect.bat"
Set LANGUAGE=GERMAN
Start "Protecting %LANGUAGE%" "E:\Build\Protect.bat"
Set LANGUAGE=FRENCH
Start "Protecting %LANGUAGE%" "E:\Build\Protect.bat"

ECHO ==STEP 3 waiting for protection to complete ==>>C:\Temp\buildlog.txt

"C:\Bin\GdPUtil.exe" -join protectENGLISH 1000
"C:\Bin\GdPUtil.exe" -join protectSPANISH 1000
"C:\Bin\GdPUtil.exe" -join protectGERMAN 1000
"C:\Bin\GdPUtil.exe" -join protectFRENCH 1000

ECHO ==STEP 3 Complete==>>C:\Temp\buildlog.txt
ECHO ==STEP 4 Starts - Code Signing protected Executables ==>>C:\Temp\buildlog.txt

Now our script works like a champ! Even though it has to do 4 times as much work, it is only slightly slower than it used to be.

Why does this matter to you?

If you have long running scripts, have a good look if they can be broken up like we did in our build script. For example, are you uploading a file to multiple FTP servers in your script? If you have enough upload bandwidth but it is the FTP servers causing the slowdown, this technique can be very helpful.

SkunkWorks - New Program, New Name, Contest, Testers.....

Hello,

We are currently working on a new program. Functionality is going great so far but we have a strange problem ;-) More on that further down below...

The Program

So far our programs WatchDirectory and WatchFTP concentrated on doing things automagically. This new program is different.
It allows you to select files (in Windows Explorer, for example), add these files to a "basket" (an icon floating above explorer) and then perform "actions" on those baskets. A few actions we are currently implementing:

• COPY - copy the files dropped into the basked to a pre-configured directory.
• MOVE - same as copy, but move the files instead.
• EMAIL - email the files to a pre-configured email address.
• FTP - upload the files to an FTP server.
• BATCH (you already knew this was coming, I guess ;-) - Run a script for each of those files - program your own action.
• YourIdea - we are always interested to hear from you.

The Name

So... if you read the above... any idea for a good name? There are a few important features the name must have:

• English - our biggest market.
• Web - the www.TheNameYouThinkOf.com name must be available.
• Attractive/Mysterious - if the name wows people who see it, making them curious, that would be great!
• Not offensive - need I explain this?
• File - it would be great if it has the word "file" in its name.
• Short - not as important as the above, but shorter names are easier to type (url in webbrowsers).

We already thought of several names (all very boring), do you know a good name? A good program name can make the difference between success or failure for a new program, so if you suggest the "winning name" to us we will compensate you for it. Beside a free copy of the program and our everlasting gratitude - a $1000 reward or 50% of the first month sales of this program - you choose!

Send an email to gert@watchdirectory.net, use the subject "SKUNKWORKS - Name". Tell us the program name and why you think it is a great name to use.

Beta Testers

It may take a month, maybe 2 months before we have the first beta ready, but we need beta testers for this new program. Beta testers receive a free copy when it is released, obviously. Interested?

Send an email to gert@watchdirectory.net, use the subject "SKUNKWORKS - Beta".

Update - July 22, 2009

We just bought the domain www.FileHalla.com (thank you Peter Wester) - any Vikings reading this ;-)
If you have a better idea, the contest is still open, probably until the end of the month.

Update - July 25, 2009

We just bought the domain www.FileViking.com (thank you Al Harberg). He didn't actually suggest this name but warned me about negative feelings the name Valhalla/Walhalla might cause.

Am I sure about FileViking? I think some ancestors of mine were slaughtered by Vikings... It has been some time ago, though...

Tweet your files with curl

This post shows how you can let WatchDirectory or WatchFTP post a new twitter message when they detect new files. Excuse me if I get some of the twitter terminology wrong - I grew up when computers were something you see in movies ;-)

Curl

This page shows that posting a message to twitter is as simple as

curl -u username:password -d status="your message here" http://twitter.com/statuses/update.json

So, we need Curl to post. I used the non-ssl version while testing. Make sure to download a Windows version from their download page.

Batch File

If you want to "tweet" when a new file is detected, create a new task and select the Run a Batch Script task (for WatchDirectory) or Run a Batch Script Action (for WatchFTP).

Let it run the following batch file (may not show correctly in all blog-readers):

@echo off
SET CURL=E:\bin\curl-7.19.5\curl.exe
SET TWUSER=TheTwitterUser
SET TWPASS=TheTwitterPassword

rem if we are called by WatchFTP, copy its variables to WD variables
rem so this script is compatible with both programs

IF "%WF_REASON%" NEQ "" (
  SET WD_REASON=%WF_REASON%
  SET WD_FILE=%WF_FILE%
)


GOTO :%WD_REASON%

:FILENEW
SET MSG="New file: %WD_FILE%"
GOTO :SendIt
:FILEDEL
SET MSG="Deleted file: %WD_FILE%"
GOTO :SendIt
:FILECHNG
SET MSG="Changed file: %WD_FILE%"
GOTO :SendIt
:FILEREN
SET MSG="Renamed file: %WD_OFILE% to %WD_FILE%"
GOTO :SendIt
:DIRNEW
SET MSG="New directory: %WD_FILE%"
GOTO :SendIt
:DIRDEL
SET MSG="Deleted directory: %WD_FILE%"
GOTO :SendIt
:DIRREN
SET MSG="Renamed directory: %WD_OFILE% to %WD_FILE%"
GOTO :SendIt


:SendIt
"%CURL%" -u %TWUSER%:%TWPASS% -d status=%MSG% http://twitter.com/statuses/update.json

Make sure to properly change the first few lines (where did you install curl, what is your twitter userid and password).

Twitter Limits

A tweet is a short message (I believe limited to about 140 characters). If your directory and filenames are long, you may need to shorten the messages a bit. For example, change %WD_FILE% to %WD_FILE_N% everywhere in the script so it will only tweet the filename (without the directory path).

Twitter also has some other limits you may hit if you call it too often.

Blogging your Files?

This post shows you how.

WatchFTP 2.2 beta

A beta version for WF 2.2 is available on our forum.

New in this beta release is that you can enter all kind of "timing" related settings for each task. Among other things, this gives you easier access to all the "secret settings" described in this blogpost.

This beta does not have the new features translated to German and Spanish yet - the new items will show in English.

To install (and keep your current settings and tasks):
* Do NOT uninstall your current WF version
* Install the beta and use all proposed settings of the installer (same directories)

WatchDirectory 4.5.9 released

We just released a new WatchDirectory version - another free upgrade for all WatchDirectory version 4 customers.

Changes in this release

• Fixed: The tip window sometimes caused the WD Control Center to crash at startup
• New: The Auto Copy plugin can now MOVE files. When you move files on the same disk, this is faster than copy followed by delete.
• New: The Folder Size plugin writes a timestamp and filename of deleted/moved files to <Task Directory>\<Task Name>\removedfiles_YYYYMMDD.csv. These files are automatically removed when they are a month old.
• New: The Folder Size plugin can check for several new "events"
• Fixed: The "Last Message" column of the WatchDirectory Control Center was not always updating properly
• Fixed: The Auto UNZIP task now supports far larger files (tested with a 2GB zip containing an 8GB file)
• Fixed: The Auto ZIP task now supports far larger files (tested up to 8GB)
• Fixed: The Auto Copy option to remove directories when they become empty in some cases also removed the monitored directory.
• New: The thumbnail plugin can now be used as a SubTask in the SubTask plugin
• New: The thumbnail plugin has a new option to prevent overwriting existing thumbnails
• New: You can set the "color" of a task. If you have a lot of tasks, this allows you to organize them in different "groups". Give a task one of 6 colors using the "right click" context menu. You can sort the tasks by color by clicking the header in the WD Control Center. You can select all tasks having the same color using the new "Colors Toolbar".
• New: When you let the email plugin combine emails, you can now set the separator between emails yourself.
• New: Use CTRL+F (or View -> Find) to find and select tasks.
• Fix: The Automatically Start any Program plugin did not properly set the environment variable WD_REASON when a new directory was detected.

How to install

Stop all your WD tasks, stop the WD Control Center. Now install the new version in the same location as your previous installed version (overwrite it).
This will keep all your settings and tasks intact.

Download

Download Here (English) or Here (German).

July Special - Free ShowSize license

When you buy 2 licenses of WatchDirectory or 2 licenses of WatchFTP during July 2009, you can get a free license of ShowSize, normally sold at $49.95.

To get your free copy of ShowSize, forward your WatchDirectory or WatchFTP order confirmation to support@watchdirectory.net. Please allow a few days to process your request.

About ShowSize

You already know that Windows Explorer does not show folder sizes in its list. ShowSize can do that and much more. See how your disk space is distributed in various folders and file types, and so on. ShowSize is the next generation software that can process terabytes of disk space and millions of files. Using ShowSize, you can also list or print folder contents in ways not possible with Windows Explorer.