Project 71: Automatic Image Caption Generation

Authors: Oihane Cantero and Julen Etxaniz

Supervisors: Oier Lopez de Lacalle and Eneko Agirre

Subject: Machine Learning and Neural Networks

Date: 20-12-2020

Objectives:

1. Implement a caption generation model that uses a CNN to condition a LSTM based language model.

2. Extend the basic caption generation system that incorporates an attention mechanism to the model.

Contents:

1. Project Description

2. Jupyter Notebook Objective 1

3. Jupyter Notebook Objective 2

4. Flickr8k Dataset and Text

5. Extracted Feature Files

6. Model Checkpoints and Images

7. Written Report

8. Presentation Slides

9. Jupyter Notebooks from Tutorials

10. Relevant Papers

References:

[1] https://arxiv.org/pdf/1411.4555.pdf

[2] https://machinelearningmastery.com/develop-a-deep-learning-caption-generation-model-in-python/

[3] https://github.com/dabasajay/Image-Caption-Generator

[4] https://github.com/yashk2810/Image-Captioning

[5] https://arxiv.org/pdf/1502.03044.pdf

[6] https://www.tensorflow.org/tutorials/text/image_captioning?hl=en

[7] https://medium.com/swlh/image-captioning-using-attention-mechanism-f3d7fc96eb0e

[8] https://www.analyticsvidhya.com/blog/2020/11/attention-mechanism-for-caption-generation/