Clay Foye

Teaser:

In this thesis, I argue that the conceptions of knowledge inside AI systems are substantially different from those described by their external representations. I want to understand the structure of knowledge in internal model representations, and how that structure is altered by different representational modalities and interfaces. This research intersects with the fields of visual studies, computational linguistics, media studies, and mechanistic interpretability. It is motivated by my interest in the relationship between knowledge and truth, and asks the following questions of AI's representations of knowledge: what is shared across various representations of knowledge, and does that shared material indicate an authentic form of knowledge? How can representations of knowledge be described as reliable or safe if no such authenticity actually exists? This thesis uses the study of epistemologies of AI systems to contribute to a larger discussion of knowledge representation, multimodality, and authenticity.

Project Description:

This thesis confronts the growing body of technical research studying the representations of knowledge in internal AI mechanisms. In particular, the field of mechanistic interpretability (MI) focuses on the arrangement of neurons and weights of transformer-based model architectures. These internal representations are stored as arrays of floating-point numbers, but MI exposes them as interfaces of texts, sounds, images, and networks. The study of this relationship between knowledge and its representations is therefore a study of multimodality. I consider a single modality or "mode" to be a set of types of relationships which are co-dependent, interpretable, not tied to specific material objects, and are distinct from other modes. In this way, modalities are only fully defined in relationship to other modalities, and only exist when perceived or fabricated by beings capable of interpretation. I can thereby define multimodality as the doomed coordination or struggle between multiple modes to constrict knowledge to a certain set of relationships. Multimodality is fabricated out of necessity, anxiety, or convenience, and thus eschews any specific, overarching explanations of its existence or application. This definition of multimodality informs three central research questions on how AI and MI challenge an understanding of knowledge.
First, this thesis asks: what is the structure of knowledge in internal AI model representations, and how is that structure altered by different modalities and interfaces? To answer this question, I will rely on the field of mechanistic interpretability (MI) and its tools and methods to describe internal model mechanisms. These methods will provide the necessary definitions and contrast between parametric knowledge, or knowledge stored in the parameters of AI systems, and representational knowledge, or knowledge stored and conveyed by multimodality. I will rely on the fields of linguistics, visual studies, semiotics, and media studies to more clearly define the idea of "representational knowledge," especially within the context of AI systems. This question will also inform the majority of my technical research: in order to study "parametric knowledge," I will apply the methods of dictionary learning through sparse autoencoders and circuit tracing via transcoders to the weights of open-source LLMs. In implementing, modifying, and studying these methods and their outputs, I hope to concretely define parametric knowledge and to specify a structure of knowledge inside AI systems.
Second, this thesis will develop the contrasted parametric and representational forms of knowledge by synthesizing them, asking: what is shared across representations of knowledge, and how does that shared information indicate an authentic form of knowledge? This research will draw upon the work of classical philosophy, theology, post-structuralism, and postcolonialism. Classical philosophy and theology will offer a metaphysics of an epistemology which presupposes an authentic form of knowledge, or truth. This thesis situates such epistemologies as being influential in the field of mechanistic interpretability as they implicitly and explicitly inform its research questions and methodology. The fields of post-structuralism and postcolonialism will offer a contrasting narrative of authenticity and the relationship between internal AI parametric knowledge and MI's representations of such knowledge. Postcolonialism in particular is well-suited to identifying and deconstructing encoded forms of hegemony and power. This research question will describe multimodality as a battleground of authenticity and competing assumptions of structures of knowledge. The question of authenticity participates at multiple levels: as motivational for the technical research and methods of mechanistic interpretability, and as catalytic for multimodal epistemologies.
Finally, this thesis will also investigate how representations of knowledge can be described as reliable or safe if no authenticity exists? The interest in questions of safety or reliability arises from the claims of mechanistic interpretability to create interfaces which improve the trustworthiness or auditability of AI systems by representing their internal mechanisms. To answer this research question, I will draw on the fields of ethics, visual studies, and media theory. Ethics offers a starting framework for motivating and defining “safety,” while visual studies and media theory can describe mechanisms of representation without leaning on authentic truth. This research question serves to combine and challenge the assumptions and outcomes of the previous two questions by confronting the project with the material consequences of AI usage.
 

Clay Foye is a Digital Humanities scholar with a degree in Computer Science from Dartmouth College and a master’s in Digital Humanities from EPFL. His past work includes computational architectural history, analysis of structures of dialogue in interactive media, and mechanistic analysis of vision-language models. He is interested in (and suspicious of) the intersection of representation and authenticity with interface and technology.