Automated contest

What to submit?

You should submit a working discovery algorithm, which can be called using a “Discover.bat” Windows batch file which takes two parameters:

  1. The full path to the discover log file, including the “.xes” extension.
  2. The full path to the model file where the discovered model should be stored, excluding any extension like “.pnml”, “.bpmn”, or “.lsk”.

Running the batch file should import the provided log file, discover a model from it, and export the discovered model to the provided model file (including the extension that matches the model).

If the results of calling your Discovery.bat file as described above is a PNML file (Petri nets), a BPMN file (BPMN diagram), or a LSK file (log skeleton), then you’re done. If not, the discovery algorithm needs to come with its own working classifier algorithm, that is, a “Classify.bat” Windows batch file, which takes three parameters:

  1. The full path to the classify log file, including the “.xes” extension.
  2. The full path to the model file which should be used to classify the classify log, excluding any extension like “.pnml”, “.bpmn”, or “.lsk”.
  3. The full path to the log file where the classified log should be stored, including the “.xes” extension.

Running the batch file should import the classify log and the model, classify the classify log using the model (adding the “pdc:isPos” attributes to the traces), and export the classified log to the provided log file.

Classification of a trace is done by adding the boolean “pdc:isPos” attribute to the trace, which should be true if the trace is classified positive (fits your model) and false if the trace is classified negative (does not fit your model).

All implemented classification algorithms (BPMN, DCR, LSK, and PNML) are available for download.

The 11 pre-existing discovery algorithms do not count as submissions, and are hence not participating in this contest. They are just there to show to current state of the discovery field. If the authors of these algorithms want to participate, they should explicitly submit the algorithm.

When to submit?

As soon as possible, but not later than August 17st. You can submit as many times as you like. Note that a new submission does not replace an old submission. Every submission counts.

How to submit?

Please let us know by an email to process.discovery.contest@gmail.com that you want to submit. We will then provide you with a link where you can upload your submission as a single (zip, tar, tar.gz) archive file. Note that the submission in the archive file should be self-contained: the algorithm(s) should be able to run without using any other software.

What feedback do I get?

The F-score results obtained using your algorithm(s). Like follows:

This shows that the Hybrid ILP Miner scores 68% over all 192 logs, and how it scores on the different configurations: It scores 83% over all 96 logs that do not involve routing tasks.

Furthermore, you will also get the classification results, like follows:

This graph shows the classifications of the Hybrid ILP Miner on the 32 different pdc_2020_02CDEFG.xes logs. Among other things, this shows that:

  • the Hybrid ILP Miner is non-deterministic (as some of the bars do not reach 1000),
  • false positives typically occur for these logs if noise is present, and
  • false negatives typically occur for these logs if OR tasks are present.

There will be similar graphs for the 00*, 01*, 10*, 11* and 12* logs.

Who wins?

The submission that results in the highest average F-score over all 192 classified test logs in the data set. In case of a tie, the earlier submission wins.

Received submissions

In total, we received 10 submissions for the automated contest. All implementations for these submissions are available for download.