|
When throttling is enabled for a watch folder endpoint,
it limits the number of watched folder jobs that can be processed
at any given time. The maximum number of jobs is determined by the
Batch Size value, also configurable in the Watched Folder endpoint.
Incoming documents in the input directory of the watched folder
will not be polled when the throttling limit has been reached. The documents
will also remain in the input directory until other watched folder
jobs have completed and another poll attempt is made. In the case
of synchronous processing, all jobs processed in a single poll will
count toward the throttling limit, even though the jobs are processed
consecutively in a single thread.
Note: Throttling does not scale with a cluster. When
throttling is enabled, the cluster as a whole will not process more
than the number of jobs specified in the Batch Size at any given
time. This limit is cluster-wide, and not specific to each node
in the cluster. For example, with a Batch Size of 2, the throttling
limit could be reached with a single node processing two jobs, and
no other nodes would poll the input directory until one of the jobs
is completed.
How throttling worksWatched Folder scans the input folder at each Repeat Interval,
picks up the number of files specified in the Batch Size, and invokes
the target service for each of these files. For example, if the
Batch Size is four, at each scan, Watched Folder will pick up four
files, create four invocation requests, and invoke the target service.
Before these requests are completed, if Watched Folder is invoked,
it will again start four jobs regardless of whether the previous
four jobs are completed.
Throttling prevents Watched Folder from invoking new jobs when
the previous jobs are not completed. Watched Folder will detect
jobs in progress and process new jobs based on the batch size minus
jobs in progress. For example, in the second invocation, if the
number of jobs completed is only three and one job is still in progress,
Watched Folder invokes only three more jobs.
Watched Folder relies on the number of files present
in the stage folder to find out how many jobs are in progress. If
files remain unprocessed in the stage folder, Watched Folder will
not invoke any more jobs. For example, if the batch size is four
and three jobs are stalled, Watched Folder will invoke only one
job in subsequent invocations. There are multiple scenarios that
can cause files to remain unprocessed in the stage folder. When
jobs are stalled, the administrator can terminate the process on
the Process Management administration page so that Watched Folder
moves the files out of the stage folder.
If the LiveCycle server goes down before Watched Folder can
invoke the jobs, the administrator can move the files out of the
stage folder. For information, see Failure points and recovery.
If the LiveCycle server is running but Watched Folder is
not running when the Job Manager service calls back, which occurs
when services do not start in the ordered sequence, the administrator
can move the files out of the stage folder. For information, see Failure points and recovery.
|
|
|
