The Origin, Design, and Applications of Multimodal Content Analysis

In the past three decades, research on multimodality has primarily focused on theoretical construction and small-scale qualitative studies, with a notable lack of empirical research grounded in large-scale corpora and rigorous methodologies. With the growing trend of interdisciplinary integration, it has become imperative for researchers to move beyond the earlier prevalence of small-scale case studies and to place greater emphasis on the generalizability of research findings. Against this backdrop, multimodal content analysis has emerged as an approach that combines quantitative content analysis with social semiotic theory. The objective of this method is to conduct structured coding and quantitative analysis of large-scale corpora. This article aims to present the research design framework of this method, which encompasses formulating quantifiable research questions, developing a multimodal coding scheme, and analyzing and interpreting the resulting quantitative multimodal data. This approach not only enriches traditional content analysis with a social semiotic perspective but also extends the applicability of multimodal discourse analysis to large-scale datasets. An application of the method to city branding short videos from Xi’an and Shenzhen demonstrates its effectiveness. The analysis not only maps the distribution of traditional versus modern attributes, but more importantly, captures the intricate ways these attributes are fused through multimodal design, which ultimately fosters a transcultural appeal in the cities’ images.