ICFHR 2016 Competition on Recognition of Handwritten Mathematical Expressions (ICFHR 2016 CROHME)


Call for Participation

We are pleased to announce the Fifth International Competition on Recognition of Handwritten Mathematical Expressions (CROHME). CROHME 2016 is being held as part of the upcoming International Conference on Frontiers in Handwriting Recognition in Shenzen, China (ICFHR).

The first four CROHME competitions have advanced the state-of-the-art for handwritten math recognition systems. The competition has produced a large, freely available corpus of handwritten formulae for academic research, along with new structural representations, evaluation metrics, and automated error analysis and visualization techniques. Results from previous competitions are published in the proceedings of ICDAR 2011/2013, ICFHR 2012/2014, and in a journal paper in IJDAR: Advancing the state of the art for handwritten math recognition: the CROHME competitions, 2011–2014.

Important dates, an overview of tasks in the competition, data formats, and other details are provided below.

Important Dates

  • April 1st: Training data available
  • April 18th: Registration deadline (go to registration page )
  • June 22th: Test data available to participants
  • June 24th 30th: extended Test result submissions are due

CROHME 2016 Tasks

  • CROHME2016-Formulas: Formula recognition from handwritten strokes (main task)
  • CROHME2016-Symbols: Classification of isolated math symbols
  • CROHME2016-Structure: (**New) Parsing formula structure from given (valid) handwritten symbols
  • CROHME2016-Matrices: (**Experimental) Expression recognition with matrices

Note: System rankings and awards will be determined by the main formula task (CROHME2016-Formulas). However, participants are encouraged to participate in whichever tasks they would like.

The main task compares complete recognition systems, which must address the key problems of symbol segmentation, symbol classification, and expression structure recognition (CROHME2016-Formulas). The Symbols and Structure sub-tasks consider symbol classification and structure recognition in isolation, providing a more controlled comparison (in particular, without complex artifacts due to segmentation errors). We will also again have an experimental task for recognition of formulas that contain matrices, which remains a very challenging problem.

The language of formulas to be recognized in Tasks 1-3 will be the same as for the last CROHME in 2014 - we will re-use the same expression grammar (Grammar IV) and symbol set (101 symbols). For the first time, the CROHME organizers provide a large corpus of over 500,00 fomulae taken from English Wikipedia for training language models (e.g. Stochastic Context-Free Grammar production probabilities).

New test sets will be provided for all tasks but CROHME2016-Structure, which will use expressions from CROHME 2014. This is necessary because symbol recognition results will need to be provided in test inputs for this sub-task.

More information about available data.