Access Today An Introduction To Information Theory: Symbols, Signals And Noise Put Together By John Robinson Pierce Compiled As EText

informative introduction to the subject, I really liked the material until the author started connecting information theory to other fields, such as: psychology, cybernetics and art, I would have appreciated a deeper dive into coding theory and telecommunication systems, Good introduction to information theory,

I'm quite certain I'd have been more benefited by this book if I had taken the time to also do the math, but it was a worthy read nonetheless.
Interesting with a lot of applications, With moments somewhat old fashioned, This book is a perfect introduction to information theory and its applications, and the significance of Shannon's discoveries, I didn't finish the book though because a few chapters were either irrelevant or too technical for me,

"Entropy is a wonderful and universal measure of amount of information, "

Chapter: The Origins of Information Theory

information theory as communication theory challenges encode/decode signal, noise bandwidth range of symbols used to encode signal channel capacity amount of signal that can reliably be transmitted in unit time

Noisychannel: The objective of communication is not to reproduce the message, but to predict the message.


Chapter: A Mathematical Model

a mathematical model to produce English text random words, unigram, bigram, trigram finitestate automata with limited capabilities ergodic source of text

Chapter: Encoding and Binary Digits

symbols and bits encode text in terms of characterscharacters gtbits per character gtbits per words considering that the average word length is.
characters or wordsbits per word assumingK vocabulary

Chapter:Entropy

Entropy is the average bits per symbol needed to encode a message, A high entropy encoding corresponds to efficient

encoding because of the less number of bits required to encode messages and thus leads to high surprise at decode time,

the fundamental theorem of noiseless channel probable messages vs, possible messages Huffman coding entropy of a message source vs, entropy of a message entropy vs, channel capacity

Chapter: Language and Meaning

language is chief form of communication but imperfect semantics ambiguity, noncompositionality pragmatics

Chapter: Efficient Encoding

text characters vs.
audio characters, pitch, volume, stress vs, video image remove redundancy

Chapter: Information Theory and Human Psychology

the entropy of a task is directly proportional to its level of difficulty however, the rate of change in difficulty slows down with increase in complexity of the task humans are good at complexity but only fair at speed Zipf's law principle of least effort

Chapter: Information Theory and Art

high entropy art sounds dull and unfamiliar vs.
low entropy art too familiar and uninteresting probably the reason why classical Indian music usesnotes and Western music usesnotes for audience to appreciate a piece of art, it should strike the right balance between order and randomness Good recap from my RF engineering days where Nyquist and Shannon's work were often applied to satcom.
. . I was hoping for a bit more on the metaphysical front but that was probably unrealistic given the chapters listed in the TOC, We are in, Pierce is showing off Information Theory to layperson like myself, You may very much appreciate his introductory discourse on mathematical models and "informed ignorance" his is humble but setting the boundaries of Shannons work, The text is warm and he dares to knit together language, cybernetics, psychology and art, Low entropy everywhere and exciting sometimes, I learned a bit and enjoyed the read, Another reviewer said it was a 'gentle thorough' introduction to the topic, I can't speak to the thorough, not my field, but it was gentle, Pierce has a sense of humor, and by golly, I believed that I could do the math if I took a real course in information theory,

What made it truly interesting to me was the date of revision,, before the big revolution in IT, InI still had to create a program in order to use a program, Reading this book was a bit nostalgic, Maybe more of a A solid notverytechnical introduction, which also explores the relation between Information Theory and other disciplines, Wellexplained and easy to read, A good intro to IT, Recommended as a first. A gentle yet solid introduction to information theory, Well written review of information theory, A bit dated, but overall demonstrates what information theory is and its applications, It seems the main purpose of this book is to provide evidence that Information / Communication Theory should be taken seriously as a real science, Perhaps because I'm reading this so many years after its writing, all I can think is, "how could it not"

The author provides a warning in the beginning that the book contains a lot of math, but because I don't have a strong math background I tend to let those parts of books like these flow over and through and pick up what I can when it gets back to the theory portions.
Nonetheless, as an audiobook, it's especially challenging to listen to recitations of long strings and formulas,

I particularly appreciate the historical reference the book provides and learning about the process involved in calculating how to move data from a sender to a receiver through a wide variety of mediums.
What exactly is an introduction At what point do you say that a book is far too technical and the material is better fit for people with intermediate experience Of course, it all depends.
With that said, I think this book does still qualify as an introduction to information theory, but it really pushes the limit, Perhaps another way to say it is that this book is better fit for students in a college course, not casual readers with a passing interest in information theory, I definitely found it interesting and I got a lot out of it, but I also consider this one of the tougher books I've read outside of my field, In total, it took me aboutmonths to read the firstpages while waiting on my clothes down at the laundromat, until I decided a couple weeks ago to just suck it up and finish it.


In the beginning of chapterPierce acknowledges the difficulty of the first chapters as necessary preparation for the rest of the book, Those first chapters are dense, technical, and boring, Probably too boring for the average reader, The sheet amount of mathematical notation was just made it so much more intimidating every time I picked it up, Eventually I decided on the strategy of ignoring whatever I didn't understand on the first pass, and that seemed to work fine, Pierce explains the ideas thoroughly, and the equations provided just offer a useful restatement of the same ideas for the mathematically inclined, I wish I'd known that when I started, He even says exactly this in the Appendix "On Mathematical Notation":

"The reader will find a fairly liberal use of mathematical notation in this book, including a number of equations.
This many incline him to say that this book is full of mathematics,

Of course it is, Communication theory is a mathematical theory, and, as this book is an exposition of communication theory, it is bound to contain mathematics, The reader should not, however, confuse the mathematics with the notation used, The book contain just as much mathematics and not include one symbol or equality sign, "

So there, do your best and get what you can, The above quote also illustrates his writing style, I wasn't sure what to make of it at first, but it grew on me, He has a dry wit, which I enjoy but I know so many people will hate,

If you have a passing interest in communication or information theory, you might want to start off with a more accessible text, If you have a fairly strong background in math or you're willing to make up the difference with effort, then this is probably the best introductory book on the topic.

Symbols, Signals and Noise lies somewhere in the aether between textbooks and popular science, It is certainly more easy to read than many examples of the former, with careful explanations in plain language of the phenomena being discussed, and intuitive explanations of any important equations or discoveries, yet it is also certainly a deeper treatment than a swathe of modern books which present only gaudy tidbits of information and some patronising examples.
Pierce stresses that his book would be empty without maths, and aims to fill in the gaps in laymen's understanding rather than skim over them even if he does omit proofs in many cases.


Most notable about the book is its readable style, Rarely does treatment of a scientific topic have this narrative pull which keeps you reading, or such a sense of an author's voice, It is a pleasure to learn from this character, whose intuitive explanations, scientific modesty and occasional spot of mild humour all work to educate you about a deep and broad topic, both in its core components as established by Shannon and in its applications to fields as diverse as linguistics, psychology and physics.


The book is not without flaws, but these are for the most part textual, The terms 'information theory' and 'communication theory' are both used, seemingly interchangeably throughout the book, with no real explanation of why this is so, There are a number of typographical errors in unfortunately critical areas numbers which are mistyped, symbols which are exchanged in the middle of important explanations, I entertained for a while that the book itself was demonstrating a form of error correction the redundancy in the text being sufficient to allow the detection and correction of errors.
In his chapter on cybernetics, Pierce discusses for a while computing and predictive programs, and his treatment here has dated quite amusingly, not that it is any fault of his that the history of computation has changed since the An Introduction to Information Theory's publication, or that problems he saw unsolved have been tackled with great energy.


Certainly a great read for anyone with an interest in the topic, and I daresay a valuable guide to understanding for an undergraduate in communications or computing, or a professional moving from another field.

.