8 processors but still waiting

Our new build system has lots of ram and a multicore hyperthreaded cpu: 4*2 processors. It is just unbelievable how responsive this Vista-64 machine is.

We are currently making some changes to the build process of WatchFTP, creating separate installers for each supported language. Previously, we created just one installer with a multilingual executable (thank you Sisulizer for a great localization product). The multilingual approach worked fine most of the time, however when a German customer living in France wants to run the English version of WatchFTP on his Spanish-language operating system it was just not possible. The operating system would select the language to display, not much you could do about it.

Future WatchFTP versions will have the desired display language build-in. Sisulizer will create language specific (German, Spanish etc) copies of the original (English) executables. The display language will only depend on the installer you download from our websites.

Steps in our build process

1. Build a "release" version of WatchFTP
   (Microsoft Visual Studio 2008 - C++ by the way)
2. Run Sisulizer to create the multilingual executables
3. Protect the core components (2 executables)
4. Code-sign the EXE and DLLs
5. Run Inno Setup to create the installer
6. Code-sign the setup file
7. Test the setup
   When we find issues, fix them and go back to step 1.
8. When all is OK, a lot of other tasks like uploading to our website etc

The most time consuming step (several minutes) was step 3 (software protection). Not really a big deal, we don't release new versions that often. However, step 7 (test setup) can cause a lot of rebuilds (start at step 1) and the delays for software protection become very annoying.

It gets Worse!!

When we implement separate installers for the different languages we support, we will have to do step 3 (protect) for each language (currently German, Spanish and English. Soon we will have a French version as well).
This will make our already long build process 4 times longer - totally unacceptable.

Our great new computer doesn't help at all because all those steps need to be sequential. We can not code-sign (step 4) before the protection (step 3) is completed. Also, the protection phase is done with a single-threaded program, even if the computer has 1000 cpus, it will only use one of them.

How we solved it

Obviously, the protection step (nr 4) needs to be paralleled. We can launch the protector as a separate process for each of the languages. A first attempt (WRONG!!) in our build batch script:

....
ECHO ==STEP 2 Complete==>>C:\Temp\buildlog.txt
ECHO ==STEP 3 Starts - Software Protection==>>C:\Temp\buildlog.txt
Set LANGUAGE=ENGLISH
Start "Protecting %LANGUAGE%" "E:\Build\Protect.bat"
Set LANGUAGE=SPANISH
Start "Protecting %LANGUAGE%" "E:\Build\Protect.bat"
Set LANGUAGE=GERMAN
Start "Protecting %LANGUAGE%" "E:\Build\Protect.bat"
Set LANGUAGE=FRENCH
Start "Protecting %LANGUAGE%" "E:\Build\Protect.bat"
ECHO ==STEP 3 Complete==>>C:\Temp\buildlog.txt
ECHO ==STEP 4 Starts - Code Signing protected Executables ==>>C:\Temp\buildlog.txt

(note: the scripts shown in this post are very simplified without error checking etcetera).
The protect.bat file is called and based on the LANGUAGE variable it will protect a specific language version of WatchFTP. The beauty of the Start command is that it will launch a separate console (command prompt) to run the protect.bat script. In the example above 4 language versions are protected simultaneously, each using their own CPU. Obviously, because of disk usage, there is some slowdown - it won't complete 4 times as fast.

A small problem...

In case you didn't realize, the above script is sort of wrong: it continues running while the 4 "Protect.bat" files are not finished yet. So it gladly goes on to Step 4 (code signing), but there isn't anything to code-sign yet. We need to wait for the 4 protect.bat processes to complete before we can continue with Step 4.

And an easy fix

We have a small command prompt program called GdPUtil (Free Download). Please read the description of the -join parameter.

We added the following lines at the end of protect.bat so it signals to the main script it is ready:

....
"C:\Bin\GdPUtil.exe" -join protect%LANGUAGE% 1000
EXIT

And changed the original build script to

....
ECHO ==STEP 2 Complete==>>C:\Temp\buildlog.txt
ECHO ==STEP 3 Starts - Software Protection==>>C:\Temp\buildlog.txt
Set LANGUAGE=ENGLISH
Start "Protecting %LANGUAGE%" "E:\Build\Protect.bat"
Set LANGUAGE=SPANISH
Start "Protecting %LANGUAGE%" "E:\Build\Protect.bat"
Set LANGUAGE=GERMAN
Start "Protecting %LANGUAGE%" "E:\Build\Protect.bat"
Set LANGUAGE=FRENCH
Start "Protecting %LANGUAGE%" "E:\Build\Protect.bat"

ECHO ==STEP 3 waiting for protection to complete ==>>C:\Temp\buildlog.txt

"C:\Bin\GdPUtil.exe" -join protectENGLISH 1000
"C:\Bin\GdPUtil.exe" -join protectSPANISH 1000
"C:\Bin\GdPUtil.exe" -join protectGERMAN 1000
"C:\Bin\GdPUtil.exe" -join protectFRENCH 1000

ECHO ==STEP 3 Complete==>>C:\Temp\buildlog.txt
ECHO ==STEP 4 Starts - Code Signing protected Executables ==>>C:\Temp\buildlog.txt

Now our script works like a champ! Even though it has to do 4 times as much work, it is only slightly slower than it used to be.

Why does this matter to you?

If you have long running scripts, have a good look if they can be broken up like we did in our build script. For example, are you uploading a file to multiple FTP servers in your script? If you have enough upload bandwidth but it is the FTP servers causing the slowdown, this technique can be very helpful.