This is a static archive of the previous Open Grid Forum GridForge content management system saved from host forge.ogf.org file /sf/wiki/do/viewPage/projects.ogsa-wg/wiki/LSFWorkflowOverview?selectedTab=versions at Sun, 06 Nov 2022 22:11:00 GMT SourceForge : View Wiki Page: LSFWorkflowOverview

Project Home

Tracker

Documents

Tasks

Source Code

Discussions

File Releases

Wiki

Project Admin

Glance

Calendar
Search Wiki Pages Project: OGSA-WG     Wiki > LSFWorkflowOverview > View Wiki Page
wiki1755: LSFWorkflowOverview
LSF supports some features for inter-job dependencies. We also have a product called Process Manager that provides calendar and event driven scheduling which includes the ability to define "flows" of jobs with dependencies between them.

For LSF, dependency features are accessed with the -w option to our submission command. Here are the details from the bsub man page:

-w 'dependency_expression'

LSF will not place your job unless the dependency expression evaluates to TRUE. If you specify a dependency on a job that LSF cannot find (such as a job that has not yet been submitted), your job submission fails.

The dependency expression is a logical expression composed of one or more dependency conditions. To make dependency expression of multiple conditions, use the following logical operators:

  • && (AND)
  • || (OR)
  • ! (NOT)

Use parentheses to indicate the order of operations, if neces- sary.

Enclose the dependency expression in single quotes (') to pre- vent the shell from interpreting special characters (space, any logic operator, or parentheses). If you use single quotes for the dependency expression, use double quotes for quoted items within it, such as job names.

In dependency conditions, job names specify only your own jobs, unless you are the LSF administrator. By default, if you use the job name to specify a dependency condition, and more than one of your jobs has the same name, all of your jobs that have that name must satisfy the test. If JOB_DEP_LAST_SUB in lsb.params is set to 1, the test is done on the job submitted most recently. Use double quotes (") around job names that begin with a number. In the job name, specify the wildcard character asterisk (*) at the end of a string, to indicate all jobs whose name begins with the string. For example, if you use jobA* as the job name, it specifies jobs named jobA, jobA1, jobA_test, jobA.log, etc.

Use the * with dependency conditions to define one-to-one dependency among job array elements such that each element of one array depends on the corresponding element of another array. The job array size must be identical.

For example:

bsub -w "done(myarrayA[*])" -J "myArrayB[1-10]" myJob2

indicates that before element 1 of myArrayB can start, element 1 of myArrayA must be completed, and so on.

You can also use the * to establish one-to-one array element dependencies with bmod after an array has been submitted.

If you want to specify array dependency by array name, set JOB_DEP_LAST_SUB in lsb.params. If you do not have this parameter set, the job will be rejected if one of your previous arrays has the same name but a different index.

In dependency conditions, the variable op represents one of the following relational operators:

  • > =
  • <
  • <=
  • ==
  • !=

Use the following conditions to form the dependency expression.

done(job_ID |"job_name" ...)

The job state is DONE.

LSF refers to the oldest job of job_name in memory.

ended(job_ID | "job_name")

The job state is EXIT or DONE.

exit(job_ID | "job_name" [,[operator] exit_code])

The job state is EXIT, and the job's exit code satisfies the comparison test.

If you specify an exit code with no operator, the test is for equality (== is assumed).

If you specify only the job, any exit code satisfies the test.

external(job_ID | "job_name", "status_text")

The job has the specified job status.

If you specify the first word of the message description (no spaces), the text of the job's status begins with the specified word. Only the first word is evaluated.

job_ID | "job_name"

If you specify a job without a dependency condition, the test is for the DONE state (LSF assumes the "done" dependency condition by default).

numdone(job_ID, operator number | *)

For a job array, the number of jobs in the DONE state satisfies the test. Use * (with no operator) to specify all the jobs in the array.

numended(job_ID, operator number | *)

For a job array, the number of jobs in the DONE or EXIT states satisfies the test. Use * (with no operator) to specify all the jobs in the array.

numexit(job_ID, operator number | *)

For a job array, the number of jobs in the EXIT state satisfies the test. Use * (with no operator) to specify all the jobs in the array.

numhold(job_ID, operator number | *)

For a job array, the number of jobs in the PSUSP state satisfies the test. Use * (with no operator) to specify all the jobs in the array.

numpend(job_ID, operator number | *)

For a job array, the number of jobs in the PEND state satisfies the test. Use * (with no operator) to specify all the jobs in the array.

numrun(job_ID, operator number | *)

For a job array, the number of jobs in the RUN state satisfies the test. Use * (with no operator) to specify all the jobs in the array.

numstart(job_ID, operator number | *)

For a job array, the number of jobs in the RUN, USUSP, or SSUSP states satisfies the test. Use * (with no operator) to specify all the jobs in the array.

post_done(job_ID | "job_name")

The job state is POST_DONE (the post-processing of specified job has completed without errors).

post_err(job_ID | "job_name")

The job state is POST_ERR (the post-processing of the specified job has completed with errors).

started(job_ID | "job_name")

The job state is:

  • RUN, DONE, or EXIT
  • PEND or PSUSP, and the job has a pre-execution command (bsub -E) that is running.

 




The Open Grid Forum Contact Webmaster | Report a problem | GridForge Help
This is a static archive of the previous Open Grid Forum GridForge content management system saved from host forge.ogf.org file /sf/wiki/do/viewPage/projects.ogsa-wg/wiki/LSFWorkflowOverview?selectedTab=versions at Sun, 06 Nov 2022 22:11:00 GMT