error

Your navigator is out-of-date

Our fonctionnalites cannot works well or create some bugs. Please use a more recent version (at least Internet Explorer 8) or an other navigator

HAMEX Handwritten and Audio Dataset of Mathematical Expressions

Contact us to get the database

Information

Categories Handwriting Recognition
Created the16 Feb 2015

Information

Hamex is a new public bimodal database of mathematical expressions. The establishment of this database is part of the DEPART project (Documents Ecrits et PAroles – Reconnaissance et Traduction). This project, founded by the Pays de la Loire French region, aims to set up a platform of tools based on two natural communication ways widely used by humans which are handwriting and speech. This platform will be used to make easier the automatic processing of languages.

The database concerned contains 4350 different expressions with different levels of complexity


Examples of mathematical expressions from the HAMEX database

Each mathematical expression is available in its online handwritten form and audio one :


Handwritten and audio mathematical expression

The vocabulary (mathematical symbols) which is considered include 74 different symbols. The symbol ’-’ refers either to ’minus sign’ or to ’fraction bar’.

Digits 0 ... 9
Latine characters a ... z W X Y
Greek characters \alpha \beta \gamma \phi \pi \theta
Operators + - \pm x / \div
Elastic symbols \sum \int \sqrt \frac
Set operators \in \forall \exist
Functions \log \sin \cos
Braces ( )
Others . , \rightarrow \infty
Equality operators = > < \neq \geq \leq

58 different writers have participated to collect the handwritten mathematical expressions dataset. Similarly, 58 French speakers have uttered the corresponding speech dataset.

In addition to the raw data (ink for the handwriting modality and audio signals for the speech modality), the ground truth of each expression with respect to each modality is also available. This ground truth is performed in both symbols and inter symbols relationships levels (fig.3) using XML formats. The INKML format is used for the ink ground-truth and Transcriber’s format (*.trs) is used for the speech ground-truth.


Example of annotation of an expression with respect to each modality

The set of mathematical expressions collected is decomposed into a validation set and an evaluation set.

Base collected Number of expressions Number of hours Number of writers/ speakers
Training 2175 6h 29
Evaluation 2175 6h 29

 

The main goal of the HAMEX database, is to be able to exploit the existing complementarity between the two modalities (handwriting and speech) to reach higher recognition rates and be able to set up more accurate systems. First attempt to perform that thanks to this database is done at a symbol level. As expected, this showed the reliability of a such procedure (table.3). Refer to [3] to know more about this experiment.

recognition process recognition rate
Speech alone 50.09%
Handwriting alone 81.55%
Fusion of the two modalities 98.04%

Source publication : Please, cite the following paper(s) in your publications if you use this database in your work

Hamex - a handwritten and audio dataset of mathematical expressions S. Quiniou, H. Mouchère, S. Peña Saldarriaga, C. Viard-Gaudin, E. Morin, S. Petitrenaud, S. Medjkoune

Conference paper11th International Conference on Document Analysis and Recognition, ICDAR 2011, Sep 2011, Beijing, China BibTex XML