Batch jobs in D365: a Russian roulette
Real world production scenarios include automated batch tasks executed in the background with some cadence: Batch processing overview – Finance & Operations. They apply to sets of documents fetched at the run time, for example to sales orders delivered, but not invoiced yet. Obviously, every day the query with the documents should return a different set with the recently despatched sales orders. It is called Late selection: Late selection in D365FO – D365Tour. So far so known.
As the complexity of the business grows, so does the complexity of the batch jobs. You’ll need jobs consisting of multiple tasks, executed one after another. It is called task dependency, or task Constraints. One of the latest unpleasant discoveries was the fact, that you cannot always rely on the sequence of steps set in the batch job: the most useful tasks, such as the SalesFormLetter_Invoice and its likes (used to update sales and purchase orders with order confirmations, delivery notes and invoices) act as orchestrators and spin off child batch tasks for simultaneous, multi-threaded sales order, purchase order, production order processing. These child tasks are not constrained (!) and get executed at a first come, first served basis. They may still run while the master task SalesFormLetter_Invoice seemingly ends and relays the baton to the next task. This leads to interference and racing conditions. You better evaluate the average execution time of the master controller task as a whole, put these controllers into different batch jobs, schedule them at a safe time distance from one another.
Heterogenous batch jobs as the one shown below pose a different challenge.
Here is the how-to instruction:
- Spin off the first task from the main menu (here: Production control > Periodic tasks > Production order status update) and choose Run in the background / Batch processing. This will be your starting point, a template for the batch job.
- Locate the batch job in the My batch jobs list, set the status to Withheld and start editing.
- Every task within a batch job needs a Class name. Now you know the class name of the first task: ProdMultiCostEstimation. To add the next task, you have to manually specify its Class name. The owners of this framework provided a lookup list, but it is slow and completely dysfunctional: it shows just a fraction of the possible executable RunBase or SysOperation classes, sometimes it crashes if there are some bad customizations in the source code. If you don’t know the class name, repeat the step 1 to reveal it.
- Add the next task, type the class name in. The first time it will take the system up to 20-30 seconds to accept and retrieve the Class description.
- Most batch tasks need a query. Click Parameters to populate it. Here you experience another nasty trait of the batch framework: the query shown is the last one you used in the system elsewhere. It is not necessarily the one really persisting at the batch task. Another user is not going to see your query criteria at all: only the owner/creator has the control. *1) *2) *3)
- On the Batch task details tab, Constraints, choose the predecessor task to build a chain. You many need to change the Expected status to Ended or error if the batch job has to continue with the next task on an error somewhere in the chain.
- Check the recurrence, remove the Alerts if needed, and put the Batch job back to Waiting.
1) Lessons learned: take a screenshot of every query and parameter, document it thoroughly to pass the maintenance to the Dynamics system administrator.
2) In certain cases a manually added task does not let edit the execution parameters and/or the query. Sometimes(e.g. on sales order updates) it helps to execute the same periodic task in the normal user session once. This is the reason why it is often hard to pre-configure the batch jobs in a “Golden Config” environment: it may need real transactions to execute against. Sometimes it wishes to be initialised from the main menu and nowhere else because badly programmed. In such a case, start your step 1 – the key task – from this critical periodic task. Should there be 2 such bitchy tasks in one batch job chain, it is over. They must be separated into different batch jobs.
3) As a really bad surprize came the fact that even the owner has no control over the query: if you use the Late selection, in a chain of nearly identical tasks as the ones below, the QUERY OF THE LAST TASK IS APPLIED TO EVERY TASK of the kind, at least in the case of the production order status updates.
You should let the bloody job run one first time, then put the batch job back on hold and adjust the queries in the tasks once more, then save and re-activate the batch job.
Dynamics 365 tips and tricks
Further reading:
SysFlushAOD: Refresh SysExtension cache in D365FO
Batch jobs in D365: a Russian roulette
Cross-company data sharing vs. Duplication in Dynamics
Refresh master data cache in D365FO
Number of records in D365
Overwrite a read-only configuration in D365FO
Copy-paste automation in D365 FO with a keyboard emulator
Make yourself an Admin in Dynamics 365 Fin/SCM
Exposing Dynamics 365 Onebox to the LAN
1 Comment
Thank for highlighting these issues. I’m glad I don’t have to find out the hard way.
Looks like Microsoft could spend some TLC making this core, critical functionality a bit smoother