Applying Machine Learning to Text Mining

Machine Learning techniques are widely adopted in the field of Text Mining. By Text Mining (also referred to as text analytics) we understand the process of deriving structured information from (written) natural language data. A preliminary stage in the  mining of texts is to group the texts into a given classification system based on  textual features. In my presentation, I would like to show a real life example of text classification from our current research activities using Machine Learning techniques.

Through our cooperation with the Federal Institute for Vocational Education and Training (Bundesinstitut für Berufsbildung, BIBB, Bonn) we have obtained access to an growing corpus of several million job advertisements, containing the texts of the job ads themselves as well as various metadata like date of publication, region, sector etc. Collections of job advertisements are a useful research object with regard to the analysis of requirements of the rapidly changing German job market. The aim of the cooperation is to extract further details from the full text of each job ad in order to to enrich the database.

As a first step we built a zone analysis application based on machine learning techniques which separates sections in the texts and classifies them into three different content classes:

  • Self description of the advertising company
  • Description of the advertised job
  • Required qualification of the potential candidates.

To evaluate whether one method is superior to other methods, we tested several thousand combinations of feature generation, feature weighting, and classification algorithms. Due to excellent evaluation scores, we can emphasize that job ads are very well suited for zone analysis applications and that the output produced by the methods could be used for further text mining applications.

[Dieser Text ist das Abstract, welches ich zum Machine Learning Day 2017 (MLD17), zu dem ich als Speaker eingeladen bin, eingereicht habe. Da die Abstracts auf der Veranstaltungsseite (noch) nicht veröffentlicht werden, habe ich es hier gepostet – wäre ja schade drum.]

Cite this article as: Jürgen Hermes: "Applying Machine Learning to Text Mining", in TEXperimenTales, 09/03/2017,

Autor: Jürgen Hermes

Arbeitet als Wissenschaftler und Geschäftsführer am Institut für Digital Humanities an der Universität zu Köln.

Ein Gedanke zu „Applying Machine Learning to Text Mining“

Schreibe einen Kommentar

Deine E-Mail-Adresse wird nicht veröffentlicht. Erforderliche Felder sind mit * markiert.

Diese Website verwendet Akismet, um Spam zu reduzieren. Erfahre mehr darüber, wie deine Kommentardaten verarbeitet werden.