creators_name: Ellerman, David
creators_id: david@ellerman.org
type: published_article
datestamp: 2011-12-23 00:46:59
lastmod: 2011-12-23 00:46:59
metadata_visibility: show
title: Counting Distinctions: On the Conceptual Foundations of Shannon's Information Theory
subjects: computation-information
subjects: history-of-philosophy-of-science
subjects: mathematics
subjects: probability-statistics
full_text_status: public
keywords: logical entropy, distinctions, Shannon entropy, logical information theory
abstract: Categorical logic has shown that modern logic is essentially the logic of subsets (or "subobjects"). Partitions are dual to subsets so there is a dual logic of partitions where a "distinction" [an ordered pair of distinct elements (u,u′) from the universe U ] is dual to an "element". An element being in a subset is analogous to a partition π on U making a distinction, i.e., if u and u′ were in different blocks of π. Subset logic leads to finite probability theory by taking the (Laplacian) probability as the normalized size of each subset-event of a finite universe. The analogous step in the logic of partitions is to assign to a partition the number of distinctions made by a partition normalized by the total number of ordered pairs |U|² from the finite universe. That yields a notion of "logical entropy" for partitions and a "logical information theory." The logical theory directly counts the (normalized) number of distinctions in a partition while Shannon's theory gives the average number of binary partitions needed to make those same distinctions. Thus the logical theory is seen as providing a conceptual underpinning for Shannon's theory based on the logical notion of "distinctions."
date: 2009-05
date_type: published
publication: Synthese
volume: 168
number: 1
publisher: Springer (Springer Science+Business Media B.V.)
pagerange: 119-149
issn: 1573-0964
related_url_url: http://www.ellerman.org/Davids-Stuff/Maths/Counting-Dits-reprint.pdf
related_url_type: author
citation: Ellerman, David (2009) Counting Distinctions: On the Conceptual Foundations of Shannon's Information Theory. [Published Article]
document_url: http://philsci-archive.pitt.edu/8967/1/Counting%2DDistinctions%2D3.pdf