vision-language-transformer

Here are 20 public repositories matching this topic...

salesforce / LAVIS

LAVIS - A One-stop Library for Language-Vision Intelligence

deep-learning salesforce image-captioning deep-learning-library vision-framework vision-and-language multimodal-deep-learning multimodal-datasets vision-language-transformer vision-language-pretraining visual-question-anwsering

Updated
Jupyter Notebook

IDEA-Research / GroundingDINO

Star

[ECCV 2024] Official implementation of the paper "Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detection"

open-world object-detection vision-language vision-language-transformer open-world-detection

Updated
Python

salesforce / BLIP

Star

PyTorch code for BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation

image-captioning visual-reasoning visual-question-answering vision-language vision-language-transformer image-text-retrieval vision-and-language-pre-training

Updated
Jupyter Notebook

AlibabaResearch / AdvancedLiterateMachinery

Star

A collection of original, innovative ideas and algorithms towards Advanced Literate Machinery. This project is maintained by the OCR Team in the Language Technology Lab, Tongyi Lab, Alibaba Group.

ocr computer-vision artificial-intelligence text-recognition document text-detection document-analysis end-to-end-ocr multimodal scene-text-recognition multimodal-deep-learning scene-text-detection vision-language document-understanding scene-text-detection-recognition document-recognition document-intelligence documentai vision-language-transformer vision-language-model

Updated
C++

henghuiding / ReLA

Star

[CVPR 2023 Highlight & IJCV 2026] GRES: Generalized Referring Expression Segmentation

multimodal-learning referring-image-segmentation referring-expression-segmentation referring-expression-comprehension vision-language-transformer cvpr2023

Updated
Python

shenyunhang / APE

Star

[CVPR 2024] Aligning and Prompting Everything All at Once for Universal Visual Perception

open-world object-detection image-segmentation referring-expression-comprehension vision-language-transformer

Updated
Python

henghuiding / Vision-Language-Transformer

Star

[ICCV2021 & TPAMI2023] Vision-Language Transformer and Query Generation for Referring Segmentation

tensorflow keras transformer vision-language referring-segmentation tpami iccv2021 vision-language-transformer

Updated
Python

haoliuhl / instructrl

Star

Instruction Following Agents with Multimodal Transforemrs

machine-learning reinforcement-learning instructions transformer flax jax instruction-following vision-language-transformer

Updated
Python

sMamooler / CLIP_Explainability

Star

code for studying OpenAI's CLIP explainability

machine-learning computer-vision gradcam-visualization model-explainability openai-clip vision-language-transformer

Updated
Jupyter Notebook

yiren-jian / BLIText

Star

[NeurIPS 2023] Bootstrapping Vision-Language Learning with Decoupled Language Pre-training

multimodal-deep-learning vision-language-transformer vision-language-pretraining

Updated
Python

unitaryai / VTC

Star

VTC: Improving Video-Text Retrieval with User Comments

comments video-understanding multimodal-deep-learning video-text-retrieval vision-language-transformer vision-language-pretraining

Updated
Python

deepmancer / vlm-toolbox

Star

Vision-Language Models Toolbox: Your all-in-one solution for multimodal research and experimentation

deep-learning transformers pytorch deep-learning-library clip multi-granularity-dataset vision-framework multimodal-learning vision-and-language multimodal-deep-learning multimodal-datasets hierarchical-classification zero-shot-classification vision-language-transformer prompt-tuning soft-prompt-tuning

Updated
Jupyter Notebook

ThomasVonWu / Awesome-VLMs-Strawberry

Star

A collection of VLMs papers, blogs, and projects, with a focus on VLMs in Autonomous Driving and related reasoning techniques.

multimodal-learning vision-language-transformer llm vlms

Updated

marialymperaiou / knowledge-enhanced-multimodal-learning

Star

A list of research papers on knowledge-enhanced multimodal learning

knowledge-graph multi-task-learning visual-reasoning visual-dialog visual-question-answering vision-and-language multimodal-deep-learning visual-storytelling multimodal-retrieval visual-grounding visual-commonsense-reasoning vision-and-language-navigation story-visualization image-text-matching vision-language-transformer image-text-retrieval vision-and-language-pre-training conditional-image-generation knowledge-enhanced-multimodal-learning knowledge-enhanced-vision-language

Updated

fork123aniket / Multi-Round-VLM-powered-Multimodal-Conversational-AI-Navigation-Bot

Star

Streamlit App Combining Vision, Language, and Audio AI Models

conversational-interface conversational-ai multimodal-learning multimodal multimodal-deep-learning multimodal-data conversational-agent conversational-bot vision-language vision-language-transformer generative-ai vision-language-model vision-language-navigation multimodal-large-language-models vision-language-learning vision-language-models internvl internvl2

Updated
Python

fork123aniket / Agentic-RAG-Story-Generation-with-Multimodal-GenAI

Star

Multimodal Agentic GenAI Workflow – Seamlessly blends retrieval and generation for intelligent storytelling

story-generation multimodal-learning multimodal multimodal-deep-learning multimodal-data vision-language vision-language-transformer generative-ai vision-language-model multimodal-large-language-models vision-language-learning generative-ai-model agentic-workflow agentic-rag agentic-ai internvl2

Updated
Python

jhakrraman / Deep-Learning-NYU-Spring-2025

Star

This reporsitory contains all the Homeworks, and Projects from the Deep Learning Course by Prof. Chinmay Hegde, in Spring 2025, at NYU.

natural-language-processing reinforcement-learning computer-vision deep-learning deep-learning-algorithms reinforcement-learning-algorithms new-york-university nyu natural-language-understanding computer-vision-algorithms deep-learning-models deep-learning-projects computer-vision-projects vision-language-transformer llms vision-language-model vision-language-learning vision-language-models

Updated

PrateekJannu / Vision-GPT

Star

Coding a Multi-Modal vision model like GPT-4o from scratch, inspired by @hkproj and PaliGemma

open-source machine-learning google artificial-intelligence gemini transformer-architecture transformer-models vision-transformer vision-language-transformer large-language-models llm large-language-model vision-language-model gpt-4o

Updated
Python

aurooj / VLM_SS

Star

Mini-batch selective sampling for knowledge adaption of VLMs for mammography.

medical-imaging miccai mammogram multimodal-learning vision-and-language multimodal-retrieval vision-language-transformer multimodal-representation-learning miccai2024 medical-vision-language-model minibatch-selective-sampling

Updated
Jupyter Notebook

atharva-naik / MMML-TermProject-VizWiz-VQA-Challenge

Star

VizWiz Challenge Term Project for Multi Modal Machine Learning @ CMU (11777)

open-source opencv natural-language-processing computer-vision image-processing pytorch question-answering open-source-project carnegie-mellon-university term-project visual-question-answering vizwiz vision-language vision-language-transformer vizwiz-vqa

Updated
Python

Improve this page

Add a description, image, and links to the vision-language-transformer topic page so that developers can more easily learn about it.

Curate this topic

Add this topic to your repo

To associate your repository with the vision-language-transformer topic, visit your repo's landing page and select "manage topics."

Learn more

URL: https://github.com/topics/vision-language-transformer