Cycle-Consistent Diverse Image Synthesis from Natural Language

Paper

Chen, Zhi and Luo, Yadan. 2019. "Cycle-Consistent Diverse Image Synthesis from Natural Language." 2019 IEEE International Conference on Multimedia & Expo Workshops (ICMEW). Shanghai, China 08 - 12 Jul 2019 United States. IEEE (Institute of Electrical and Electronics Engineers). https://doi.org/10.1109/ICMEW.2019.00085

Paper/Presentation Title	Cycle-Consistent Diverse Image Synthesis from Natural Language
Presentation Type	Paper
Authors	Chen, Zhi and Luo, Yadan
Journal or Proceedings Title	Proceedings of 2019 IEEE International Conference on Multimedia & Expo Workshops (ICMEW)
Journal Citation	pp. 459-464
Number of Pages	6
Year	2019
Publisher	IEEE (Institute of Electrical and Electronics Engineers)
Place of Publication	United States
ISBN	9781538692141
Digital Object Identifier (DOI)	https://doi.org/10.1109/ICMEW.2019.00085
Web Address (URL) of Paper	https://ieeexplore.ieee.org/document/8795022
Web Address (URL) of Conference Proceedings	https://ieeexplore.ieee.org/xpl/conhome/8777006/proceeding
Conference/Event	2019 IEEE International Conference on Multimedia & Expo Workshops (ICMEW)
Event Details	2019 IEEE International Conference on Multimedia & Expo Workshops (ICMEW) Parent IEEE International Conference on Multimedia and Expo Delivery In person Event Date 08 to end of 12 Jul 2019 Event Location Shanghai, China Rank B B
Abstract	Text-to-image translation has become an attractive yet challenging task in computer vision. Previous approaches tend to generate similar, or even monotonous, images for distinctive texts and overlook the characteristics of specific sentences. In this paper, we aim to generate images from the given texts by preserving diverse appearances and modes of the objects or instances contained. To achieve that, a novel learning model named SuperGAN is proposed, which consists of two major components: an image synthesis network and a captioning model in a Cycle-GAN framework. SuperGAN adopts the cycle-consistent adversarial training strategy to learn an image generator where the feature distribution of the generated images complies with the distribution of the generic images. Meanwhile, a cycle-consistency loss is applied to constrain that the caption of the generated images is closed to the original texts. Extensive experiments on the benchmark dataset Oxford-flowers-102 demonstrate the validity and effectiveness of our proposed method. In addition, a new evaluation metric is proposed to measure the diversity of synthetic results.
Keywords	Image synthesis; image captioning; generative adversarial networks; cycle-consistency loss
Contains Sensitive Content	Does not contain sensitive content
ANZSRC Field of Research 2020	460304. Computer vision
Public Notes	Files associated with this item cannot be displayed due to copyright restrictions.
Byline Affiliations	University of Queensland

Permalink -

https://research.usq.edu.au/item/zyx16/cycle-consistent-diverse-image-synthesis-from-natural-language

8
total views
0
total downloads
3
views this month
0
downloads this month

Export as

Related outputs

Continual Text-to-Video Retrieval with Frame Fusion and Task-Aware Routing

Zhao, Zecheng, Chen, Zhi, Huang, Zi, Sadiq, Shazia and Chen, Tong. 2025. "Continual Text-to-Video Retrieval with Frame Fusion and Task-Aware Routing." 48th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR '25). Padua, Italy 13 - 18 Jul 2025 United States. Association for Computing Machinery (ACM). https://doi.org/10.1145/3726302.3729936

Dynamic Target Distribution Estimation for Source-Free Open-Set Domain Adaptation

Yu, Zhiqi, Liao, Zhichao, Li, Jingjing, Chen, Zhi and Zhu, Lei. 2025. "Dynamic Target Distribution Estimation for Source-Free Open-Set Domain Adaptation." 39th AAAI Conference on Artificial Intelligence (AAAI 2025). Philadelphia, Pennsylvania, United States 25 Feb - 04 Mar 2025 United States. Association for the Advancement of Artificial Intelligence (AAAI). https://doi.org/10.1609/aaai.v39i21.34380

Snap and diagnose: An advanced multimodal retrieval system for identifying plant diseases in the wild

Wei, Tianqi, Chen, Zhi and Yu, Xin. 2024. "Snap and diagnose: An advanced multimodal retrieval system for identifying plant diseases in the wild." 6th ACM International Conference on Multimedia in Asia (MMAsia '24). Auckland, New Zealand 03 - 06 Dec 2024 United States. Association for Computing Machinery (ACM). https://doi.org/10.1145/3696409.3700293

DiPEx: Dispersing Prompt Expansion for Class-Agnostic Object Detection

Lim, Jia Syuen, Chen, Zhuoxiao, Baktashmotlagh, Mahsa, Chen, Zhi, Yu, Xin, Huang, Zi and Luo, Yadan. 2024. "DiPEx: Dispersing Prompt Expansion for Class-Agnostic Object Detection." 38th Annual Conference on Neural Information Processing Systems (NeurIPS 2024). Vancouver, Canada 10 - 15 Dec 2024 Canada.

Towards Cost-Efficient Federated Multi-agent RL with Learnable Aggregation

Zhang, Yi, Wang, Sen, Chen, Zhi, Xu, Xuwei, Funiak, Stano and Liu, Jiajun. 2024. "Towards Cost-Efficient Federated Multi-agent RL with Learnable Aggregation." 28th Pacific-Asia Conference on Knowledge Discovery and Data Mining (PAKDD 2024). Taipei, Taiwan 07 - 10 May 2024 Springer. https://doi.org/10.1007/978-981-97-2253-2_14

Benchmarking In-the-wild Multimodal Disease Recognition and A Versatile Baseline

Wei, Tianqi, Chen, Zhi, Huang, Zi and Yu, Xin. 2024. "Benchmarking In-the-wild Multimodal Disease Recognition and A Versatile Baseline." 32nd ACM International Conference on Multimedia (MM '24). Melbourne, Australia 28 Oct - 01 Nov 2024 United States. Association for Computing Machinery (ACM). https://doi.org/10.1145/3664647.3680599

Secondary analysis of newly diagnosed type 2 diabetes subgroups and treatment responses in the MARCH cohort

Wang, Weihao, Li, Xinyao, Chen, Fei, Wei, Ran, Chen, Zhi, Li, Jingjing, Qiao, Jingtao, Pan, Qi, Yang, Wenying and Guo, Lixin. 2024. "Secondary analysis of newly diagnosed type 2 diabetes subgroups and treatment responses in the MARCH cohort." Diabetes and Metabolic Syndrome: Clinical Research and Reviews. 18 (1). https://doi.org/10.1016/j.dsx.2023.102936

Optimizing taxi route planning based on taxi trajectory data analysis

Yang, Xinyi, Chen, Zhi and Luo, Yadan. 2023. "Optimizing taxi route planning based on taxi trajectory data analysis." 34th Australasian Database Conference (ADC 2023). Melbourne, Australia 01 202 - 03 Nov 2023 Switzerland . Springer. https://doi.org/10.1007/978-3-031-47843-7_4

Cal-SFDA: Source-Free Domain-adaptive Semantic Segmentation with Differentiable Expected Calibration Error

Wang, Zixin, Luo, Yadan, Chen, Zhi, Wang, Sen and Huang, Zi. 2023. "Cal-SFDA: Source-Free Domain-adaptive Semantic Segmentation with Differentiable Expected Calibration Error." 31st ACM International Conference on Multimedia (MM '23). Ottawa, Canada 29 Oct 202 - 03 Nov 2023 United States. Association for Computing Machinery (ACM). https://doi.org/10.1145/3581783.3611808

Zero-Shot Learning by Harnessing Adversarial Samples

Chen, Zhi, Zhang, Pengfei, Li, Jingjing, Wang, Sen and Huang, Zi. 2023. "Zero-Shot Learning by Harnessing Adversarial Samples." 31st ACM International Conference on Multimedia (MM '23). Ottawa, Canada 29 Oct 202 - 03 Nov 2023 United States. Association for Computing Machinery (ACM). https://doi.org/10.1145/3581783.3611823

FluMA: An Intelligent Platform for Influenza Monitoring and Analysis

Chen, Xi, Chen, Zhi, Wang, Zijian, Qui, Ruihong and Luo, Yadan. 2022. "FluMA: An Intelligent Platform for Influenza Monitoring and Analysis." 33rd Australasian Database Conference (ADC 2022). Sydney, Australia 02 - 04 Sep 2022 Switzerland . Springer. https://doi.org/10.1007/978-3-031-15512-3_12

GSMFlow: Generation Shifts Mitigating Flow for Generalized Zero-Shot Learning

Chen, Zhi, Luo, Yadan, Wang, Sen, Li, Jingjing and Huang, Zi. 2022. "GSMFlow: Generation Shifts Mitigating Flow for Generalized Zero-Shot Learning." IEEE Transactions on Multimedia. 25, pp. 5374-5385. https://doi.org/10.1109/TMM.2022.3190678

Application of Novel Subgroups of Chinese Inpatients with Diabetes Based on Machine Learning Paradigm

Wang, Weihao, Chen, Zhi, Wang, Sen, Chen, Fei, Deng, Mingqun, Fan, Qi and Guo, Lixin. 2022. "Application of Novel Subgroups of Chinese Inpatients with Diabetes Based on Machine Learning Paradigm." Diabetes and Metabolic Syndrome: Clinical Research and Reviews. 16 (7). https://doi.org/10.1016/j.dsx.2022.102556

Pixel Exclusion: Uncertainty-aware Boundary Discovery for Active Cross-Domain Semantic Segmentation

You, Fuming, Li, Jingjing, Chen, Zhi and Zhu, Lei. 2022. "Pixel Exclusion: Uncertainty-aware Boundary Discovery for Active Cross-Domain Semantic Segmentation." 30th ACM International Conference on Multimedia (MM '22). Lisbon, Portugal 10 - 14 Oct 2022 United States. Association for Computing Machinery (ACM). https://doi.org/10.1145/3503161.3548079

Distinguishing Unseen from Seen for Generalized Zero-shot Learning

Su, Hongzu, Li, Jingjing, Chen, Zhi, Zhu, Lei and Lu, Ke. 2022. "Distinguishing Unseen from Seen for Generalized Zero-shot Learning." 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). New Orleans, LA, United States 18 - 24 Jun 2022 United States. IEEE (Institute of Electrical and Electronics Engineers). https://doi.org/10.1109/CVPR52688.2022.00773

Semantics Disentangling for Generalized Zero-Shot Learning

Chen, Zhi, Luo, Yadan, Qui, Ruihong, Wang, Sen, Huang, Zi, Li, Jingjing and Zhang, Zheng. 2022. "Semantics Disentangling for Generalized Zero-Shot Learning." 2021 IEEE/CVF International Conference on Computer Vision (ICCV). Montreal, Canada 10 - 17 Oct 2021 United States. IEEE (Institute of Electrical and Electronics Engineers). https://doi.org/10.1109/ICCV48922.2021.00859

Local graph convolutional networks for cross-modal hashing

Zhang, Yudong, Wang, Sen, Lu, Jianglin, Chen, Zhi, Zhang, Zheng and Huang, Zi. 2021. "Local graph convolutional networks for cross-modal hashing." 29th ACM International Conference on Multimedia (MM '21). 20 - 24 Oct 2021 United States. Association for Computing Machinery (ACM). https://doi.org/10.1145/3474085.3475346

Domain Adaptive Semantic Segmentation Without Source Data

You, Fuming, Li, Jingjing, Zhu, Lei, Chen, Zhi and Huang, Zi. 2021. "Domain Adaptive Semantic Segmentation Without Source Data." 29th ACM International Conference on Multimedia (MM '21). 20 - 24 Oct 2021 United States. Association for Computing Machinery (ACM). https://doi.org/10.1145/3474085.3475482

Application of New International Classification of Adult‐Onset Diabetes in Chinese Inpatients with Diabetes Mellitus

Wang, Weihao, Pei, Xiaobei, Zhang, Lina, Chen, Zhi, Lin, Dong, Duan, Xiaoye, Fan, Jingwen, Pan, Qi and Guo, Lixin. 2021. "Application of New International Classification of Adult‐Onset Diabetes in Chinese Inpatients with Diabetes Mellitus." Diabetes - Metabolism: Research and Reviews. 37 (7). https://doi.org/10.1002/dmrr.3427

Mitigating Generation Shifts for Generalized Zero-Shot Learning

Chen, Zhi, Luo, Yadan, Wang, Sen, Qui, Ruihong, Li, Jingjing and Huang, Zi. 2021. "Mitigating Generation Shifts for Generalized Zero-Shot Learning." 29th ACM International Conference on Multimedia (MM '21). 20 - 24 Oct 2021 United States. Association for Computing Machinery (ACM). https://doi.org/10.1145/3474085.3475258

CausalRec: Causal Inference for Visual Debiasing in Visually-Aware Recommendation

Qui, Ruihong, Wang, Sen, Chen, Zhi, Yin, Hongzhi and Huang, Zi. 2021. "CausalRec: Causal Inference for Visual Debiasing in Visually-Aware Recommendation." 29th ACM International Conference on Multimedia (MM '21). 20 - 24 Oct 2021 United States. Association for Computing Machinery (ACM). https://doi.org/10.1145/3474085.3475266

Entropy-Based Uncertainty Calibration for Generalized Zero-Shot Learning

Chen, Zhi, Huang, Zi, Li, Jingjing and Zhang, Zheng. 2021. "Entropy-Based Uncertainty Calibration for Generalized Zero-Shot Learning." 32nd Australasian Database Conference (ADC 2021). Dunedin, New Zealand 29 Jan - 05 Feb 2021 Switzerland . Springer. https://doi.org/10.1007/978-3-030-69377-0_12

Rethinking Generative Zero-Shot Learning: An Ensemble Learning Perspective for Recognising Visual Patches

Chen, Zhi, Wang, Sen, Li, Jingjing and Huang, Zi. 2020. "Rethinking Generative Zero-Shot Learning: An Ensemble Learning Perspective for Recognising Visual Patches." 28th ACM International Conference on Multimedia (MM '20). Seattle, United States 12 - 16 Oct 2020 United States. Association for Computing Machinery (ACM). https://doi.org/10.1145/3394171.3413813

Canzsl: Cycle-Consistent Adversarial Networks for Zero-Shot Learning from Natural Language

Chen, Zhi, Li, Jingjing, Luo, Yadan, Huang, Zi and Yang, Yang. 2020. "Canzsl: Cycle-Consistent Adversarial Networks for Zero-Shot Learning from Natural Language." 2020 IEEE Winter Conference on Applications of Computer Vision (WACV). Snowmass, United States 01 - 05 Mar 2020 United Stated. IEEE (Institute of Electrical and Electronics Engineers). https://doi.org/10.1109/WACV45572.2020.9093610