PhilSci Archive

Methods for identifying emergent concepts in deep neural networks

Räz, Tim (2023) Methods for identifying emergent concepts in deep neural networks. Patterns, 4. pp. 1-7.

[img]
Preview
Text
PIIS266638992300106X.pdf - Published Version
Available under License Creative Commons Attribution Non-commercial No Derivatives.

Download (2MB) | Preview

Abstract

The present perspective discusses methods to detect concepts in internal representations (hidden layers) of deep neural networks (DNNs), such as network dissection, feature visualization, and testing with concept activation vectors (TCAV). I argue that these methods provide evidence that DNNs are able to learn non-trivial relations between concepts. However, the methods also require users to specify or detect concepts via (sets of) instances. This underdetermines the meaning of concepts, making the methods unreliable. The problem could be overcome, to some extent, by systematically combining the methods and by using synthetic datasets. The perspective also discusses how conceptual spaces—sets of concepts in internal representations—are shaped by a trade-off between predictive accuracy and compression. I argue that conceptual spaces are useful, or even necessary, to understand how concepts are formed in DNNs but that there is a lack of method for studying conceptual spaces.


Export/Citation: EndNote | BibTeX | Dublin Core | ASCII/Text Citation (Chicago) | HTML Citation | OpenURL
Social Networking:
Share |

Item Type: Published Article or Volume
Creators:
CreatorsEmailORCID
Räz, Timtim.raez@unibe.ch
Keywords: deep neural networks concepts interpretability internal representation image classification TCAV network dissection feature visualization
Subjects: General Issues > Data
Specific Sciences > Artificial Intelligence > Machine Learning
General Issues > Technology
Depositing User: Tim Räz
Date Deposited: 14 Sep 2023 19:38
Last Modified: 14 Sep 2023 19:38
Item ID: 22544
Journal or Publication Title: Patterns
Publisher: Cell Press
DOI or Unique Handle: https://doi.org/10.1016/j.patter.2023.100761
Subjects: General Issues > Data
Specific Sciences > Artificial Intelligence > Machine Learning
General Issues > Technology
Date: 9 June 2023
Page Range: pp. 1-7
Volume: 4
URI: https://philsci-archive.pitt.edu/id/eprint/22544

Monthly Views for the past 3 years

Monthly Downloads for the past 3 years

Plum Analytics

Altmetric.com

Actions (login required)

View Item View Item