Paper Circle
Grounding as Feedback: Vision-Language Alignment via Sampling-based Visual Projection
How your CV stacks up
Upload your CV to see how well it fits this job role
?%
Visual Language Model and Conflicting Cues
A Visual Language Model (VLM) learns a joint understanding of image and text and generates text based on this understanding. Yet when multiple visual cues within an image conflict—such as a written word and its ink color—we do not fully understand how the model decides which signal to prioritize.
A classical psychological paradigm to study how conflicting cues affect decision is the Stroop test, where participants are shown words in incongruent ink colors (e.g., the word "red" written in blue) and are instructed to report the ink color rather than read the word. We adapt the Stroop paradigm to VLMs and study how conflicting cues in the written word or ink color influence model behavior.
Reasons to use Rodeo
Start with a chat, not a search bar
Grad scheme, placement, apprenticeship? Not sure what you want yet — that's fine. Your agent talks it through with you and turns "I have no idea" into a shortlist.
It searches the market for you
Every day your agent scans the market matching roles against what actually matters to you, not just keywords on a CV.
Only hits
No noise. No "maybe this fits." Just roles with a clear explanation of why they're right — and where to focus when applying.
Applying the Stroop test on a range of contrastive and generative VLMs suggests the models favor textual cues over color when text and color conflict. Analyzing the representation of the two cue types suggests that text cues in images are more salient than the color cues. This difference in saliency also translates to different intervention success in steering the VLMs: we found that it is easier to steer the embedding to make the model favor text cues than color cues.
Get help with your application
Your very own career expert that helps elevate your application to the next level.
Overall, using the Stroop test, our findings suggest VLMs, similar to humans, are biased to "read" an image rather than to "see," and the saliency of the two cue types is reflected in their embedding space. We will release our dataset and code to support future research upon acceptance.
“It took my CV and asked me questions relevant to understanding what kind of jobs to suggest for me. Suggestions were almost perfect. Jobs were exactly what I’ve been looking for.”
Jessica, London
Skills