Proposal for a shorter EndToEndId in pain.001, or a 3-letter hash for RecId

Proposal for a shorter EndToEndId in pain.001, or a 3-letter hash for RecId

As mentioned in the recent blog Matching own payments in Modern bank reconciliation, the <EndToEndId> element generated by the Electronic Reporting configuration for ISO20022 Credit Transfers is built from the AP payment journal line voucher + RecId of the journal line + a 2-digit counter if multiple attempts are made to generate the same pain.001 file:

				
					<EndToEndId>APP-0036644-5637312576-02</EndToEndId>
				
			

In my opinion, the RecId (the “5637312576” part in the example) is unnecessary. Consider the arguments:

  1. The voucher number is unique in every journal line, unless the Voucher series parameter at the Journal name is set to Manual or One voucher number only, or the Number allocation at posting  is activated. All three setups are truly exotic in Dynamics 365 for Finance implementations for a number of reasons:
  2.  The One voucher number only option contradicts the recommended GL master setting [Dis]allow multiple transactions within one voucher. An attempt to post is going to produce 2 error messages for every journal line – “There can only be one vendor or customer transaction per voucher.” and “There can only be one bank account transaction per voucher.
  3. The Number allocation at posting defies the purpose: in the unposted journal, the system assigns temporary voucher numbers, which are then replaced at posting. The SEPA pain.001 file is made before posting, and its EndToEndId is going to hold the temporary voucher number:  <EndToEndId>Temp001-5637312576-02</EndToEndId>. But such a line cannot be reconciled with the Modern bank reconciliation (see Matching own payments in Modern bank reconciliation), since the bank transaction will receive a different, the final voucher number on posting.
  4. The RecId has a low entropy: it is usually monotonously rising in a payment journal, since the lines are created all at once by the Payment proposal. The number of lines is typically below 1000, as a longer journal takes increasingly longer to post. Consequently, just the last 3-4 digits of the RecId are significant.
  5. In certain geographies, the EndToEndId may not be that long (India: 16 characters only). Truncating from the right will remove the important attempt counter, and the bank upload will fail on a second attempt. Truncating from the left will corrupt the voucher number: now the bank statement reconciliation will fail.

I suggest replacing CONCATENATE(@.PaymentIdentifications.EndToEndIdentification, "_", LEFT("00", 2-LEN(@.'$paymentEnumerator')), @.'$paymentEnumerator')  in model.Payments.$PreprocessedEndToEndId 

…with CONCATENATE(REPLACE(@.PaymentIdentifications.EndToEndIdentification,”-\d+$”,””,true),”_”, LEFT(“00”, 2-LEN(@.’$paymentEnumerator’)), @.’$paymentEnumerator’)

This eliminates the last group of digits together with the trailing “-” from the EndToEndIdentification element, and the result is going to be like this:

				
					<CdtTrfTxInf>
        <PmtId>
          <InstrId>APP-0036644-2026</InstrId>
          <EndToEndId>APP-0036644-02</EndToEndId>
        </PmtId>
				
			

Shorten the RecId with a 3-letter hash “ABC”

If for some reason the RecId as a unique element must be preserved (I would not know, why, but let’s assume), then it may be reduced from a long 64-integer to a pseudo-unique 3 letter combination, derived from the last 4 digits of the RecId. These 4 digits are shuffled and dispersed, then the resulting integer value is converted into a base-26 number, encoded by capital latin letters:

				
					str input = REPLACE(model.Payments.PaymentIdentifications.EndToEndIdentification, "^.*-", "", true)
int last4 = INTVALUE(RIGHT(@.input, 4))
int mixed = @.last4*73 + 41                             ' prime numbers *salt*
int mod17576 = @.mixed-INTVALUE(@.mixed/17576)*17576    ' 17576 = 26³

int c1v  = INTVALUE(@.mod17576/676)  ' 676 = 26²
int rem1 = @.mod17576 - @.c1v*676
int c2v  = INTVALUE(@.rem1/26)       ' same as div(26)
int c3v  = @.rem1 - @.c2v*26

str output = MID("ABCDEFGHIJKLMNOPQRSTUVWXYZ", @.c1v+1, 1)&
             MID("ABCDEFGHIJKLMNOPQRSTUVWXYZ", @.c2v+1, 1)&
             MID("ABCDEFGHIJKLMNOPQRSTUVWXYZ", @.c3v+1, 1)
				
			

The input is the EndToEndIdentification stripped of the preceding voucher number up to the last “-” i.e. the RecId at the end. The result is a short deterministic string at the place of the longer RecId number:

<EndToEndId>APP-0036644UUT29</EndToEndId>

<EndToEndId>APP-0036645UXO01</EndToEndId>