Ellerman, David (2013) An Introduction to Logical Entropy and its Relation to Shannon Entropy. [Preprint]

PDF (To be published in International Journal of Semantic Computing)
IntroLogicalEntropystd1.pdf  Published Version Download (290kB) 
Abstract
The logical basis for information theory is the newly developed logic of partitions that is dual to the usual Boolean logic of subsets. The key concept is a "distinction" of a partition, an ordered pair of elements in distinct blocks of the partition. The logical concept of entropy based on partition logic is the normalized counting measure of the set of distinctions of a partition on a finite setjust as the usual logical notion of probability based on the Boolean logic of subsets is the normalized counting measure of the subsets (events). Thus logical entropy is a measure on the set of ordered pairs, and all the compound notions of entropy (join entropy, conditional entropy, and mutual information) arise in the usual way from the measure (e.g., the inclusionexclusion principle)just like the corresponding notions of probability. The usual Shannon entropy of a partition is developed by replacing the normalized count of distinctions (dits) by the average number of binary partitions (bits) necessary to make all the distinctions of the partition.
Export/Citation:  EndNote  BibTeX  Dublin Core  ASCII/Text Citation (Chicago)  HTML Citation  OpenURL 
Social Networking: 
Item Type:  Preprint  

Creators: 


Keywords:  partition logic, logical entropy, Shannon entropy, information as distinctions  
Subjects:  Specific Sciences > Computation/Information General Issues > Formal Learning Theory Specific Sciences > Mathematics 

Depositing User:  David Ellerman  
Date Deposited:  16 Jan 2014 21:18  
Last Modified:  16 Jan 2014 21:18  
Item ID:  10230  
Subjects:  Specific Sciences > Computation/Information General Issues > Formal Learning Theory Specific Sciences > Mathematics 

Date:  2013  
URI:  https://philsciarchive.pitt.edu/id/eprint/10230 
Monthly Views for the past 3 years
Monthly Downloads for the past 3 years
Plum Analytics
Actions (login required)
View Item 