Project Overview
Building a novel brain-guided convolutional neural network augmented with large language models to study how knowledge influences perception
Department(s)
Neuroscience
Psychological and Brain Sciences
Computer Science
Abstract
The neural representation of information in any environment is not a static pattern, but instead undergoes multiple transformations over time, and supports the use of different types of information with differing task demands (i.e., how someone modifies their behavior to achieve a particular action or goal). However, exactly how task-relevant information is built up and subsequently used is only vaguely understood. To tackle that problem, most of the efforts in systems neuroscience have focused on modeling (via convolutional neural networks) the neural representations of visual information without considering what the person is trying to do with that information. While such approaches have provided insight into how the neural code of information supports recognition, they fall short in two crucial ways. 1) They do not consider the behavioral goals of the person in any given context (i.e., how they are using the information), and 2) visual information by itself is isolated from the relative object and context associations that exist in our enviroments, which are typically captured in large language models (LLMs). This project therefore seeks to construct a novel convolutional neural network (CNN) where the convolutional layers were independently guided by brain responses (measured via EEG) at different time points as participants enegage in different tasks. Further, as the different brain-guided CNN layers learn task relevant features, how those features are weighted and passed to subsequent layers will be guided by different LLM associations that best align with task relevant neural responses. This novel brain-guided network will be trained on EEG data obtained from participants engaged in different tasks in order to gain novel insight into how knowledge and task demands shape the neural code that supports perception.
Student Qualifications
Must be highly motivated, diligent, relaible, and most of all, intellectually curious. Applicants that have taken NEUR 170 or PSYC 275, and COSC 101 will be given priority.
Number of Student Researchers
2 students
Project Length
10 weeks
Applications open on 11/03/2024 and close on 03/06/2025