NeSyVQA: Neurosymbolic Visual Qestion Answering via Scene Graph Enrichment

Requirements

• Ubuntu 18.04
• CUDA 10.1
• Python 3.7
• PyTorch 1.4
• KGTK 0.5

Downloads

• GQA dataset
• VG dataset
• CSKG embeddings
• Faster RCNN pretrained network
• SGG pretrained network
• Preprocessed datasets, precomputed embeddings amnd pretrained networks are available in SSG-VQA/downloads

Code

• Scene Graph Generation and Enrichment with Evaluation
• Visual Question Answering with Evaluation

Directories

• Scene Graph Generation and Enrichment: SSG
• Visual Question Answering: VQA
• Generated scene graphs of all images (in Eval_IO/vg/0_images) in the dataset are saved to Eval_IO/vg/1_pred_scene_graphs
• Enriched scene graphs are saved to Eval_IO/vg/2_enriched_scene_graphs
• VQA results are saved to Eval_IO/vg/3_vqa