Wednesday, March 16, 2011

Use of VisualWorks’ Polycephaly at GeoKnowledge

At GeoKnowledge we use Cincom VisualWorks Smalltalk to develop GeoX, a decision support solution for play, prospect and field assessment in the upstream petroleum industry.

We have around 150 unit tests that check the static quality of our code. Examples of tests include text spelling, verification of correctly defined class hierarchies, correct use of pragmas, etc. We also run a subset of the “Code Critics” rules included in VisualWorks. If there is a problem in code, we will see if it can be statically checked and then write a code quality test.

We have made our own tool to execute the tests, write the result to a window, load new code and re-execute:

GeoX “Test Runner”

The process of running the 150 tests takes about an hour in a single image. We wanted to run the tests faster to detect errors quicker. To do this we used Polycephaly.

We made a small extension to Polycephaly to let it process a set of operations (a job) using a pool of virtual machines. (The two methods we added are found below.) The extension allows starting a hard-coded number of virtual machines, and let those execute a set of tasks. When a virtual machine finishes a task, it is given the next task in the job. All machines are kept busy during the execution of the job.

By using a fixed-size pool of Polycephaly worker images, we limit the number of virtual machines started. This is important; if we started 150 images to run our unit tests in parallel, we would use a lot of resources (memory in particular) without being able to execute the job faster. This approach is similar to how Erlang does its thread handling.

Using our extensions to Polycephaly, the tool does the following:
  1. Load newest code.
  2. Save the image.
  3. Set up a pool, of for example 4, Polycephaly virtual machines.
  4. Using the pool, send tasks to the virtual machines. Each task is the instruction to perform a single test. Answer is a Boolean indicating failure/success.
  5. When job is finished report result to user interface.
  6. Wait for new code published, restart process.
Below are the results of running the 150 tests, using 4 Polycephaly virtual machines. “Original time” refers to executing the job using a single image:

Intel Quad CPU Q8200 (4 cores, 4 threads)
Total time went down to 28% of original time.
This is near linear scaling.

Running virtualized on an Intel Core i5 750 Microsoft Hyper-V (4 cores, 4 threads)
Total time went down to 32% of original time.
We do not understand why the tests do not scale as good using this setup, but we suspect virtualization hurts performance.

Intel Core i5 661 (2 cores, 4 threads)
Total time went down to 43% of original time.
This shows how raw execution of Smalltalk code benefits more from using “true” cores, than Intel threads.

Instance-side code extensions to Polycephaly.VirtualMachines
doActions: actions
"Do all actions using the receiver’s machine pool.
To reduce the total execution time, smaller tasks should be at the end of the action collection.
Answer an array with the result of each action."

^self doActionsAndArguments: (actions collect: [:each | each -> Array new])

doActionsAndArguments: actionsAndArguments
"Do all actions using the receiver’s machine pool. Argument actionsAndArguments is a dictionary where each association holds the action block and its argument collection.
To reduce the total execution time, smaller tasks should be at the end of the action collection.
Answer an array with the result of each action."

| machinesReady answerSemaphore dronesSemaphore answer |

machinesReady := Array new: self machines size withAll: true.
answer := Array new: actionsAndArguments size.
dronesSemaphore := Semaphore new.
answerSemaphore := Semaphore new.
dronesSemaphore initSignalsTo: self machines size.
actionsAndArguments doWithIndex: [:each :index | | indexOfReadyMachine |
dronesSemaphore wait.
indexOfReadyMachine := machinesReady indexOf: true.
machinesReady at: indexOfReadyMachine put: false.
at: index
put: ((self machines at: indexOfReadyMachine) do: each key withArguments: each value).
machinesReady at: indexOfReadyMachine put: true.
dronesSemaphore signal.
answerSemaphore signal] fork].
actionsAndArguments size timesRepeat: [answerSemaphore wait].

No comments: