Some additional web links relevant to Condor

• An interesting link describing how to work with Java and Condor.
• Different examples with their corresponding submission files.
• Other interesting site describing how to use Condor.
• Another interesting link for rookie Condor users.
  

Enabling the "Parallel" universe in Condor

If you have used our script (check the last reference) for deploying Condor then you need modify your local configuration files (condor_config.local) at master and worker nodes as follows:

Master:

UNUSED_CLAIM_TIMEOUT = 0
MPI_CONDOR_RSH_PATH = $(LIBEXEC)
ALTERNATE_STARTER_2 = $(SBIN)/condor_starter
STARTER_2_IS_DC = TRUE
SHADOW_MPI = $(SBIN)/condor_shadow

Worker Nodes:

DedicatedScheduler = "DedicatedScheduler@PASTE_SUBMIT_NODE_NAME_HERE" 
STARTD_ATTRS = $(STARTD_ATTRS), DedicatedScheduler
SUSPEND = False
CONTINUE = True
PREEMPT = False
KILL = False
WANT_SUSPEND = False
WANT_VACATE = False
RANK = Scheduler =?= $(DedicatedScheduler)
MPI_CONDOR_RSH_PATH = $(LIBEXEC)
CONDOR_SSHD = /usr/sbin/sshd
CONDOR_SSH_KEYGEN = /usr/bin/ssh-keygen
STARTD_EXPRS = $(STARTD_EXPRS), DedicatedScheduler

# Dedicated Node.

START=TRUE

In addition, I attach a submission file for running parallel jobs.

should_transfer_files = Yes
when_to_transfer_output = ON_EXIT_OR_EVICT
universe = parallel
executable = /bin/hostname
+ParallelShutdownPolicy = "WAIT_FOR_ALL"
machine_count = 3
log        = hostLog
Output     = hostOut.$(Node).$(Process)
Error      = hostErr.$(Node).$(Process)
queue

The boldfaced line forces that Condor will wait until every node in the parallel job has completed.

References

• MPI applications within Condor
• An easy way to deploy Condor
  

My journey with libguestfs

When I work with Virtual Machines (VMs) many times I define/create a template hard disk (HD). This template could be later attached to a VM and then modified for attending specific needs. For instance, I have created an Ubuntu-based VM with just the OpenSSH service, ready to go. On top of it, I  can deploy different services and applications, such as:

• web servers
• monitoring tools
• hacking tools
• and so on

One of the hardest part of all this process, regards with modifying files inside the filesystem. One option is attach the hard disk to one VM, boot the VM up, log into the running VM and modify the required files. Very handy, uh?

I found the 'libguestfs' project. It provides a set of tools that allows you to view and modify files inside VMs. Check the project home page for more details and features.

I'm installing it on my Fedora 13 box. Since I would love to do some hacks in Java, executed the following command 'yum install libguestfs-java-devel'. Around 19 packages were necessary for installing the aforementioned package, so I accepted and voila!.

Next entry, I will talk about how to hack some virtual hard disks!

Open in terminal - Nautilus

I'm a user of the Gnome window manager therefore I use its file manager, Nautilus.
Sometimes, I'm looking for a specific directory and need to start working there from the command line.
By default, Nautilus doesn't provide any tool for doing that but there is a package named 'nautilus-open-terminal' to enable that feature.
For Fedora systems run: sudo yum install nautilus-open-terminal.

For using xterm instead of terminal, check the second item in the 'References' section.

References:

• Nautilus-open-terminal: quick launch
• Using xterm
  

dos2unix in Ubuntu

I installed a JeOS appliance and it lacks of the 'dos2unix' utility. I found this site which explains how to install the aforementioned utility, aptitude install tofrodos.

Looks like this and other utilities are not installed in the last Ubuntu release, 10.04.

Many-task applications and divisible load theory

Recently, I found a paper about a new kind of applications named 'Many-task applications'. Those applications fill a gap between high performance computing and high throughput computing. Better say, those applications regard with bulk of tasks may be static or dynamic, homogeneous or heterogeneous, loosely coupled or tightly coupled, [1].
Divisible load theory offers another approach to the scheduling problem. DLT, for short, has been widely studied and applied to different distributed and large scale scenarios with important restrictions about the nature of the applications to be scheduled. However, DLT provides a close and optimal solution when some restrictions are observed.
It's important, then, to analyse the nature and characteristics of many-task applications in such a way that DLT approach can be successfully applied under circumvented  circumstances.


[1] Many-task computing for grids and supercomputers. I. Raicu, I. Foster, Y. Zhao, 2008.
[2] Scheduling many-task workloads on supercomputers; dealing with trailing tasks. T. Armstrong, Z. Zhang, D. Katz, M. Wilde, I. Foster, 2010.

Working with Qemu and GNS3

QEMU is an open source machine emulator. It is used by GNS3 to emulate PCs into network topologies defined by the user.
Why do we need a PC into a network topology? Linux-based PCs can be used as firewalls and routers. For our purposes we are interested in to deploy a virtual machine with firewall and router capabilities.
GNS3, our network simulation tool, exhibits valuable features but it lacks of consolidated documentation. As follow, I will provide some links for deploying a PC QEMU host in GNS3.

• Downloading and configuring a QEMU-based image. The configuration steps are performed into GNS3.
• Some tricks for running the terminal of the PC. The prior link suggests to modify some flags for configuring the VM image into GNS3, however the author of this link suggests no modify any flag if you require to access via terminal the PC host in GNS3 (Of course, we need it). I tested and worked BUT looks like there are connections problems with the deployed PC with other network equipments in the network topology.
  
• Download the more recent version of GNS3. I'm working with GNS3 version 0.7.3 but the link above suggest to work with the development version of GNS3. 

Stay tuned!

focus-follow-mouse

A desirable and useful behaviour under Linux GUIs is to get the focus of a window in which the mouse pointer is over. In Gnome, this behaviour is not enable by default, in fact, it isn't even configurable with the default administration/configuration tools. Two steps are then required to enable this feature

1. Install the "control-center-extra" package. As root, execute "yum install control-center-extra"
2. When the package is successfully installed, go to "System -> Preferences -> Window" and select the item "Select windows when the mouse moves over them"

References: Fedora forum