关键词: 多模态AI, 非物质文化遗产, 数智技术, 图像识别

Abstract:

[Purpose/Significance] Against the background of the national cultural digitalization strategy and the rapid development of artificial intelligence, the protection and transmission of intangible cultural heritage is shifting from documentation-oriented digital recording to long-term management, knowledge-based organization, and active utilization. Intangible cultural heritage is characterized by living transmission, embodied practice, oral communication, contextual dependence, and community participation. Its key knowledge is often distributed across images, audio recordings, videos, textual documents, field notes, tools, materials, ritual spaces, and the experiences of bearers. Traditional digitization methods, which mainly focus on media storage and platform display, are no longer sufficient to support systematic protection, evidence-based research, or practice-oriented transmission. Multimodal artificial intelligence provides a possible technical route for addressing this challenge. By jointly processing image, audio, video, text, three-dimensional, and contextual data, multimodal AI can improve the organization, association, retrieval, interpretation, and reuse of intangible cultural heritage resources. It is especially valuable for documenting procedural knowledge, representing cultural contexts, processing multilingual and dialectal materials, and supporting knowledge services based on traceable evidence. This study aims to clarify how multimodal AI can empower the scientific safeguarding and digital-intelligent transmission of intangible cultural heritage, and to propose an implementation path that is both technically feasible and culturally appropriate. [Method/Process] From the perspective of knowledge organization and data governance in library and information science, this paper first clarifies the connotations and relationships between intangible cultural heritage, scientific safeguarding, digital-intelligent transmission, multimodal AI, and multimodal knowledge data. On this basis, it reviews relevant research progress concerning image and video recognition, speech transcription, text recognition, multimodal retrieval, knowledge graph construction, and generative AI applications. Instead of merely listing existing platforms, the study selects representative cases of resource aggregation, speech and oral tradition collection, and motion recording practice for comparative analysis. These cases correspond to three important scenarios in intangible cultural heritage digitization: the aggregation and public access of heritage resources, the processing of oral and linguistic materials, and the documentation of embodied and procedural knowledge. Through these cases, the paper examines the applicability and limitations of the proposed implementation logic. Particular attention is paid to the fact that most intangible cultural heritage projects do not possess large-scale annotated datasets. Therefore, the study avoids assuming a conventional "large dataset-model training-automatic recognition" route, and instead emphasizes a gradual mechanism based on minimum viable datasets, pretrained models, human-machine collaborative annotation, expert verification, and continuous feedback. [Results/Conclusions] The study proposes that multimodal AI applications for intangible cultural heritage should follow an integrated technical path covering data collection and organization, recognition and transcription, structural processing, cross-media association and retrieval, knowledge organization and knowledge-base construction, and evidence-based question answering and content generation. Data construction should move beyond the storage of media files and include procedural steps, key actions, tool materials, operational rules, field conditions, ritual contexts, and source information. Such data should be traceable and locatable at the segment level. It should also be reusable for later retrieval, citation, verification, and teaching. In the technical processing stage, image and video recognition can support the extraction of patterns, objects, scenes, actions, and process fragments; speech recognition can transform oral narratives, interviews, chants, and dialect materials into searchable texts; text recognition can convert scanned documents, manuscripts, genealogies, and historical records into structured resources. Multimodal retrieval and knowledge graphs can further align different carriers of information and organize them around persons, places, projects, procedures, tools, and events. Generative AI should not be used as unconstrained content production. Instead, it should be embedded in knowledge-base-supported services, with generated answers, explanations, teaching materials, and public-facing content accompanied by verifiable source locations such as page ranges, timestamps, or database records. This can reduce the risks of distortion, cultural misinterpretation, and unauthorized reuse. The paper further argues that, under low-resource conditions, AI applications in intangible cultural heritage should begin with low-risk auxiliary tasks such as speech transcription, video segmentation, preliminary image classification, similar-resource retrieval, metadata completion, and tag recommendation. Outputs generated by AI systems should be reviewed by heritage experts, community participants, and bearers, and the corrected results should be fed back into datasets for model refinement and rule optimization. In this sense, Multimodal AI does not replace bearers or communities; rather, it provides a set of explainable, traceable, and iterative tools for scientific documentation, knowledge organization, transmission support, and protection management. Future research should further explore data standards for procedural knowledge, evaluation methods for culturally sensitive AI outputs, authorization mechanisms, community participation, and sustainable governance models for digital-intelligent intangible cultural heritage protection.

Key words: multimodal AI, intangible cultural heritage, digital-intelligence technology, image recognition