Be-or-Not Prompt Enhanced Hard Negatives Generating For Memes Category Detection
Paper
Cui, Jian, Li, Lin and Tao, Xiaohui. 2023. "Be-or-Not Prompt Enhanced Hard Negatives Generating For Memes Category Detection." 2023 IEEE International Conference on Multimedia and Expo (ICME). Brisbane, Australia 10 - 14 Jul 2023 United Sates. https://doi.org/10.1109/ICME55011.2023.00038
Paper/Presentation Title | Be-or-Not Prompt Enhanced Hard Negatives Generating For Memes Category Detection |
---|---|
Presentation Type | Paper |
Authors | Cui, Jian, Li, Lin and Tao, Xiaohui |
Journal or Proceedings Title | Proceedings of 2023 IEEE International Conference on Multimedia and Expo (ICME) |
Journal Citation | 2023-July, pp. 174-179 |
Number of Pages | 6 |
Year | 2023 |
Place of Publication | United Sates |
ISBN | 9781665468916 |
Digital Object Identifier (DOI) | https://doi.org/10.1109/ICME55011.2023.00038 |
Web Address (URL) of Paper | https://ieeexplore.ieee.org/abstract/document/10219786 |
Web Address (URL) of Conference Proceedings | https://ieeexplore.ieee.org/xpl/conhome/10219544/proceeding |
Conference/Event | 2023 IEEE International Conference on Multimedia and Expo (ICME) |
Event Details | 2023 IEEE International Conference on Multimedia and Expo (ICME) Parent IEEE International Conference on Multimedia and Expo Delivery In person Event Date 10 to end of 14 Jul 2023 Event Location Brisbane, Australia |
Abstract | Memes are one of the most popular social media in online disinformation campaigns. Their creators often use a variety of rhetoric and psychological skills to achieve the purpose of misinformed audiences. These characteristics lead to the unsatisfactory performance of memes category detection tasks, such as predicting propaganda techniques, being harmful or not, and so on. To this end, we propose a novel memes category detection model via Be-or-Not Prompt Enhanced hard Negatives generating (BNPEN). Firstly, our BNPEN is reformulated into a contrastive learning-based image-text matching (ITM) task through category-padded prompt engineering. Secondly, we design the be-or-not prompt templates to keep the writing style of memes and create hard negative image-text pairs. Finally, our negatives generating can alleviate the negative-positive-coupling (NPC) effects in contrastive learning, thus improving the image-text matching quality. Conducted on two public datasets, experimental results show that our BNPEN is better than the off-the-shelf multi-modal learning models in terms of F1 and Accuracy measures. © 2023 IEEE. |
Keywords | Be-or-Not Prompt; Hard Negatives; Contrastive Learning |
ANZSRC Field of Research 2020 | 460508. Information retrieval and web search |
Public Notes | Files associated with this item cannot be displayed due to copyright restrictions. |
Byline Affiliations | Wuhan University, China |
University of Southern Queensland |
Permalink -
https://research.usq.edu.au/item/z275w/be-or-not-prompt-enhanced-hard-negatives-generating-for-memes-category-detection
46
total views0
total downloads5
views this month0
downloads this month