Gemälde Dataset - AI-Enhanced Art Historical Descriptions This dataset contains 224x224 images and relative metadata extracted from the MIDAS XML of the Catalogue of the Photographic Collection of the Bibliotheca Hertziana enriched with AI-generated prose texts. The dataset is limited to photographs of objects classified as painting (Gemälde), and has been processed using Google Gemma 2 9B Instruct large language model on the KISSKI HPC cluster of the GWDG. Scripts to process the data on KISSKI have been elaborated with Claude Code in Virtual Studio Code. Dataset Overview Source Data: Original dataset: gemalde.tsv (19,051 rows) Extracted from: MIDAS XML format (combined.xml) Institution: Photographic Collection. Bibliotheca Hertziana - Max Planck Institute for Art History Photographic Collection Catalogue: Fotothek der Bibliotheca Hertziana Output: Enriched metadata: TSV files with AI-generated German and English descriptions 224x224 images downloaded from IIIF Image Api of the Photographic Collection Processing Pipeline 1. Data Extraction Source data was extracted with gemalde.xql from MIDAS XML format combined.xml containing structured art historical metadata including: Object titles and descriptions (textobj, textfoto) Artist information (aob30) Location data (aob26, aob28) Dating and provenance Image references (a8540) Images Download 224x224 images downloaded in advance from the IIIF Service based on gemalde.tsv. The script processing for AI Text Enrichment from the metadata checks that the image has been downloaded, so the output data has a 100% certainty of having a matching image. 17,657 images downloaded from 19,051 rows. This is due to known missing digital images. The dataset corresponds to published data and each row contains the licence and accessibility of the single image, date of creation and last update of the catalogue object. 2. AI Text Generation Model Used: Name: Google Gemma 2 9B Instruct Parameters: 9 billion Quantization: FP16 (no quantization) Context window: 8,192 tokens License: Gemma Terms of Use Processing Workflow: Input cleaning: Removal of numeric codes, normalization of Unicode characters Paragraph generation: German text from structured metadata Translation: German → English Categories processed: paragraph foto DE/EN - Photograph description paragraph obj DE/EN - Object/artwork description paragraph verwalter DE/EN - Collection/custodian information paragraph standort DE/EN - Location information AI Prompts Used Paragraph Generation Prompt Convert the following structured information into a coherent text in German. The text contains field data that should be transformed into flowing prose while preserving all information.

IMPORTANT: - Write a MAXIMUM of 2 paragraphs - Do NOT include any URLs or web links - Do NOT include reference codes or numerical codes - Do NOT add any comments or explanations - Only output the paragraph text itself

Field: {field_name} Text: {cleaned_text}

German text (maximum 2 paragraphs): Example Input: Field: textobj Text: Bildnis Filippo Neri Hl. Filippo Neri geboren 1515 Florenz gestorben 1595 Rom Priester Ordensgründer Gründer Oratorium Kongregation des Oratoriums Example Output: Filippo Neri, geboren 1515 in Florenz und gestorben 1595 in Rom, war ein Priester und bedeutender Ordensgründer. Er gründete das Oratorium und die Kongregation des Oratoriums, die bis heute eine wichtige Rolle in der katholischen Kirche spielen. Translation Prompt Translate the following German text to English. Preserve the meaning and style as much as possible.

IMPORTANT: - Do NOT include any URLs or web links in the translation - Do NOT include reference codes starting with "bh" followed by numbers - Do NOT include numerical codes like 08012353 - Do NOT add any comments or explanations - Only output the translated text itself

German text: {text}

English translation: Example Translation: Input (DE): Filippo Neri, geboren 1515 in Florenz und gestorben 1595 in Rom, war ein Priester und bedeutender Ordensgründer.

Output (EN): Filippo Neri, born 1515 in Florence and died 1595 in Rome, was a priest and important founder of a religious order. KISSKI Cluster Resources Hardware Configuration GPU: NVIDIA A100 (80GB VRAM) Architecture: Ampere Tensor Cores: 432 FP16 Performance: ~312 TFLOPS Memory Bandwidth: 2 TB/s Allocation per job: GPUs: 1× A100 CPUs: 4 cores RAM: 64 GB Time limit: 6 hours per job Job Array Configuration Array setup: Total jobs: 38 (indices 0-37) Chunk size: 500 rows per job Parallel jobs: 10 simultaneous Total rows processed: 19,000 (rows 0-18,999) Performance Metrics AI operations per row: 4 paragraph generations (foto, obj, verwalter, standort) 4 translations (DE → EN) Total: 8 LLM inference calls per row Resource consumption: GPU hours: ~125 GPU hours total (38 jobs × 3.3 hours) Model size in memory: ~18 GB (FP16) Peak VRAM usage: ~25 GB per job Output Structure data_gemalde/ ├── enriched_data/ │ ├── data_0-499.tsv # Rows 0-499 │ ├── data_500-999.tsv # Rows 500-999 │ ├── data_1000-1499.tsv # Rows 1000-1499 │ └── ... ├── images/ │ ├── {image_id_1}.jpg # IIIF thumbnail (224×224) │ ├── {image_id_2}.jpg │ └── ... └── README.md # This file Output Fields Each TSV file contains the original metadata plus AI-generated fields: Original fields: All fields from gemalde.tsv including: a8540 - Image ID (BILDDATEI-NR.) textobj - Original object text textfoto - Original photo text aob26, aob28, aob30 - Relations etc. Generated fields: paragraph foto DE - German description of photograph paragraph foto EN - English translation paragraph obj DE - German description of object/artwork paragraph obj EN - English translation paragraph verwalter DE - German description of collection paragraph verwalter EN - English translation paragraph standort DE - German description of location paragraph standort EN - English translation Technical Details Model Configuration model = AutoModelForCausalLM.from_pretrained( "google/gemma-2-9b-it", device_map="cuda", torch_dtype=torch.float16, local_files_only=True ) Generation Parameters Paragraph generation: max_new_tokens: 500 temperature: 0.7 top_p: 0.9 do_sample: True Translation: max_new_tokens: 500 temperature: 0.3 (lower for more deterministic translation) top_p: 0.9 KISSKI Documentation Main documentation: https://docs.hpc.gwdg.de/ GPU partitions: https://docs.hpc.gwdg.de/how_to_use/compute_partitions/gpu_partitions/ Account types: https://docs.hpc.gwdg.de/start_here/account_types/ Data Usage & Citation Source Institution: Bibliotheca Hertziana - Max Planck Institute for Art History Website: https://www.biblhertz.it/ Fotothek: https://fotothek.biblhertz.it/ AI Processing: Model: Google Gemma 2 9B Instruct Infrastructure: KISSKI (GWDG Göttingen) Processing date: November 2024 License: Please refer to the Bibliotheca Hertziana for source data licensing terms. Quality Notes AI-generated texts are meant to enhance discoverability and accessibility Generated descriptions may contain inaccuracies or interpretations Always refer to original structured metadata (textobj, textfoto) for authoritative information Translations preserve meaning but may not capture all nuances of art historical terminology Generated: November 2024 Processing location: KISSKI HPC Cluster, GWDG Göttingen Contact: pietro.liuzzo@biblhertz.it