All non-MPI jobs on slave nodes are executed through bpsh. Though bpsh is really a simple wrapper for the bproc system, it's use and functionality will be documented separatly here, since the functionality of bproc itself falls more in the scope of advanced programming than everyday use.
The bproc system does it's best to preserve the running environment of the user's shell on the slave node. This include the CWD. BPROC will set the CWD of a migrated process to the same pathname as the cwd on the master if it exists. Note that this is an equivalent NAME only and so may or may not actually be the same file system. Bproc does not (and really cannot) take the possibility of an alternate mount point for an NFS file system into account. Depending on whether the user's CWD is on a shared file system, and whether an equivalent (perhaps shared) structure exists on the slave node, the CWD on the slave node will be in one of 3 states:
If on a shared file system (e.g. /home/user where slaves NFS mount /home as /home), the CWD will be the very same directory.
If on a parallel but unshared structure (e.g. /scratch/user exist on master and slave but are NOT shared through NFS or similar, the CWD on the slave will be the same NAME but will be unshared.
Should no equivalent path exist on the slave, CWD will be /
A primary point of the bproc system is to allow the master to contain all configuration, libraries, and program binaries yet allow processes to be run seamlessly on slave nodes. To that end, it implements process migration. As a wrapper for bproc, bpsh uses that migration capability. For an example, let's say you run
bpsh 4 myprog <mydata
Loads the process into memory much like any shell would.
Migrates the process to node 4 in the same cwd if possible
Arranges through bpmaster and bpslave for stdin, stdout, and stderr to be redirected back to the master.
Transfers control to the program
The contents of mydata are available to the process from stdin as usual.
All environment variables are transfered to the slave node by virtue of the process migration. That is, since the process's memory image is transferred, so to are the environment variables it had access to.
stdin, stdout, and stderr are re-directed back to the master. Thus, any command line redirections through the <, >, |, etc will refer to files on the master node, without regard to their possible existence on the slave node. In general, this is The Right Thing. On occasion, it is not (such as when you want output from a long running process to be held in scratch space on the slave to be copied back later). In these rare cases, it will be necessary to place the binary in a shared directory, and wrap it's invocation inside a shell such as:
bpsh bash -c '/shared_space/myprog >/scratch/myout1'
Future versions of bpsh may address this limitation if interest is sufficient.
Bproc unifies the pid space of the cluster. Thus, the process will see all processes it has permission to see anywhere in the cluster. It may freely signal any such process from the slave node and will have the same effect as if it were running on the master.