> For the complete documentation index, see [llms.txt](https://www.oja-guide.de/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://www.oja-guide.de/navigation/overview-challenges.md).

# Overview - Challenges

{% hint style="info" %}
If you have identified any challenges not listed here, please let us know by making a pull request in our GitHub Repository or contact us directly.
{% endhint %}

## Data Collection

* Where to find OJA data? [Data Collection](/steps/data-collection.md#data-sources-and-oja-landscaping)
* How to collect OJA data? [Data Collection](/steps/data-collection.md#web-scraping)
* How to store OJA data? [Data Collection](/steps/data-collection.md#job-posting-data-schema)

## Data Enrichment

* How to segment a job ad? [Data Enrichment](/steps/data-enrichment.md#text-segmentation)
* How to identify duplicates? [Data Enrichment](/steps/data-enrichment.md#identifying-duplicates)
* How to extract occupations? [Data Enrichment](/steps/data-enrichment.md#normalising-job-titles)
* How to extract skills and competences? [Data Enrichment](/steps/data-enrichment.md#extracting-skills)

## Evaluation and Quality Control

* How to evaluate extraction and classification algorithms? [Evaluation and Quality Control](/steps/evaluation-and-quality-control.md#evaluation-metrics)
* How to create a gold standard for evaluation? [Evaluation and Quality Control](/steps/evaluation-and-quality-control.md#gold-standard-annotation-and-quality)

## Taxonomies and Ontologies

* Which taxonomies are there (ISCO, ESCO, KLDB, etc.)? [Taxonomies and Ontologies](/steps/taxonomies-and-ontologies.md#taxonomies-for-online-job-ad-analysis)
* How to develop a taxonomy? [Taxonomies and Ontologies](/steps/taxonomies-and-ontologies.md#developing-a-taxonomy)
* How to evaluate a taxonomy? [Taxonomies and Ontologies](/steps/taxonomies-and-ontologies.md#data-standards-for-taxonomies-and-ontologies)

## Dataset Curation and Representativity Analysis

* How to deal with duplicates? [Dataset Curation and Representativity Analysis](/steps/dataset-curation-and-representativity-analysis.md#deduplication)
* How to validate an OJA dataset/sample? [Dataset Curation and Representativity Analysis](/steps/dataset-curation-and-representativity-analysis.md#representativity-analysis)
* What to report?  [Evaluation and Quality Control](/steps/evaluation-and-quality-control.md#reporting-results)