The logic of information, or the logical theory of information, considers the information content of logical signs — everything from bits to books and beyond — along the lines initially developed by Charles Sanders Peirce. In this line of development the concept of information serves to integrate the aspects of logical signs that are separately covered by the concepts of denotation and connotation, or, in roughly equivalent terms, by the concepts of extension and comprehension.
Peirce gave early expression to his ideas about the "laws of information" in his lectures "On the Logic of Science" at Harvard University (1865) and the Lowell Institute (1866). Here is one of the starting points:
Thus, let us commence with the term colour; add to the comprehension of this term, that of red. Red colour has considerably less extension than colour; add to this the comprehension of dark; dark red colour has still less [extension]. Add to this the comprehension of non-blue — non-blue dark red colour has the same extension as dark red colour, so that the non-blue here performs a work of supererogation; it tells us that no dark red colour is blue, but does none of the proper business of connotation, that of diminishing the extension at all.
Thus information measures the superfluous comprehension. And, hence, whenever we make a symbol to express any thing or any attribute we cannot make it so empty that it shall have no superfluous comprehension. I am going, next, to show that inference is symbolization and that the puzzle of the validity of scientific inference lies merely in this superfluous comprehension and is therefore entirely removed by a consideration of the laws of information. (C.S. Peirce, "The Logic of Science, or, Induction and Hypothesis" (1866), CE 1, 467).