Cryptography

Cryptography (from Greek kryptós, "hidden", and gráphein, "to write") is generally understood to be the study of the principles and techniques by which information can be translated into a "garbled" version that is difficult for an unauthorized person to read, while still allowing the intended reader to convert the resulting gobbledygook back into the original information. In fact, cryptography covers rather more than merely encryption and decryption. It is, in practice, a specialized branch of information theory with substantial additions from other branches of mathematics, and from such sources as Machiavelli, Sun Tzu, and Clausewitz[?]. The term cryptology has sometimes been used instead of cryptography for this field; but there is some tension between these two lexigraphic schools. There is also some tension between fans of two spellings of cypher (the alternate is cipher). In English, the cypher spelling has historical pride of place. This and related articles in the Wikipedia are often revised by those with strong opinions on the spelling question.

Unsurprisingly, the study of hiding messages from others has been accompanied by the study of how to read such messages when one is not the intended receiver; this area of study is called cryptanalysis. People involved in such work, and with cryptography in general, are known as cryptographers (or for those in the other school, cryptologists).

The original information being sent from one person (or organization) to another is usually called the plaintext. Encryption is the plaintext-to-garble conversion, and decryption is the garble-to-plaintext conversion. A major class of encryption technique is called encoding (yielding codetext), after which the receiver decodes the codetext. The other major class is called enciphering (yielding, naturally, cyphertext), after which the receiver decyphers the cyphertext. The exact operation of the encryption and decryption, for all schemes with any pretense to security, is controlled by one or more keys.

Overview: goals

Cryptography has four main goals, though they are nearly always concealed beneath a blanket of confusing 'marketing speak' in commercial products. And behind a fog of rumor and myth as well. Examining any proposed crypto system with these basic functions in mind, and ignoring the marketing blather, will be a very useful exercise for those interested in cryptography in the real world. They are:

1. message confidentiality: Only the authorised recipient should be able to extract the contents of the message from its encrypted form. In addition, it should not be possible to obtain information about the message contents (such as a statistical distribution of certain characters) as this makes cryptanalysis easier.
2. message integrity: The recipient should be able to determine if the message has been altered during transmission.
3. sender authentication: The recipient should be able to identify the sender, and verify that the purported sender actually did send the message.
4. sender non-repudiation: The sender should not be able to deny sending the message.

Not all cryptographic systems or algorithms achieve all of the above goals, or are even intended to. Poorly designed, or poorly implemented, crypto systems achieve them only by accident or bluff or lack of interest on the part of the opposition, and users can and regularly do reduce even well designed and implemented crypto systems to the security equivalent of Swiss cheese. But even with well designed, well implemented, and properly used crypto systems, some goals aren't practical (or desirable) in some contexts. For example, the sender of the message may want to be anonymous, or the system may be intended for an environment with limited computing resources, or confidentiality might not matter.

In addition, some confusion may arise in a crypto system design regarding whom we are referring to when speaking of the "sender" or "recipient"; some examples for real crypto systems in the modern world include:

1. a computer program on a local system,
2. a computer program on a 'nearby' system which 'provides security services' for users on other nearby systems,
3. or -- what most people assume is "obviously" meant -- a human being (usually understood as one 'at a keyboard' to actively send or receive). Even in such cases, the human does not actually encrypt or sign or decrypt or authenticate anything in modern cryptographic systems. At most, when all is right in the world, the user instructs a computer program to encrypt or sign or decrypt and authenticate, or ... and it does so, properly and securely. This buffering of human action from actions which are presumed (without much consideration) to have 'been done by a human' is a source of problems in crypto system design, implementation, and use. Such problems are often quite subtle and correspondingly obscure. Generally, even to practicing cryptographers with knowledge, skill, and good engineering sense.

When confusion on these points is present (at the design stage, during implementation, or by a user after installation), unintended failures in reaching each of the stated goals can occur quite easily, often without notice to any human involved, and even given perfect algorithms, superb and provably secure system design, and error free implementation. Such failures are most often due to extra-cryptographic issues; each such failure demonstrates that good algorithms, good protocols, good system design, and good implementation do not alone, nor in combination, provide 'security'. Instead, careful thought is required regarding the entire system design and its use in actual production -- too often, this is absent or insufficient in practice with real-world crypto systems.

Although cryptography has a long and complex history, it wasn't until the 19th century that it developed anything more than ad hoc approaches to either cryptanalysis (eg, Charles Babbage's Crimean War era work on mathematical cryptanalysis of polyalphabetic cyphers, repeated publicly rather later by the Prussian Kasiski) or encryption (eg, Auguste Kerckhoffs' writings in the later 19th century). An increasingly mathematical trend accelerated up to World War II (notably in William F. Friedman's application of statistical techniques to cryptography and in Marian Rejewski's initial break into the German Army's version of the Enigma system). Both cryptography and cryptanalysis have become far more mathematical since WWII. Even then, it has taken widely available computers, and the Internet, to bring effective cryptography into common use by anyone other than national governments or similarly sized enterprises.

Classical Cryptography

The earliest known use of cryptography is found in non-standard hieroglyphics on monuments from Egypt's Old Kingdom (ca 4000 years ago). These are not thought to be serious attempts at secret communications, however, but rather to have been attempts at mystery, intrigue, or even amusement for literate onlookers. Each of which has been, intermittently, still another use of cryptography, or of something that looks (impressively if misleadingly) like it. Later, Hebrew scholars made use of simple substitution ciphers (such as the Atbash cipher) beginning perhaps around 500 to 600 BCE. Cryptography has a long tradition in religious writing likely to offend the dominant culture or political authorities. Perhaps the most famous is the 'Number of the Beast' from the book of Revelations in the Christian New Testament. 666 is almost certainly a cryptographic (ie, encrypted) way of concealing a dangerous reference; many scholars believe it's a way of referring to Rome, or Nero, (and so to Roman policies of persecution of Christians) that would be understood by the initiated (who 'had the codebook') and yet be safe (or at least somewhat deniable and so less dangerous) if it came to the attention of those authorities. At least for orthodox Christian writing, the need for such concealment ended with Constantine's conversion and the adoption of Christianity as the official religion of the Empire.

The Greeks of Classical times are said to have known of cyphers (eg, the scytale transposition cypher claimed to have been used by the Spartan military). Herodutus tells us of secret messages physically concealed beneath wax on wooden tablets or as a tattoo on a slave's head concealed by regrown hair (see secret writing; these are not properly examples of cryptography). The Romans certainly did (eg, the Caesar cipher and its variations). There is ancient mention of a book about Roman military cryptography (especially Julius Caesar's); it has been, unfortunately, lost. Cryptography became (secretly) important still later as a consequence of political competition and religous analysis. For instance, in Europe during and after the Renaissance, citizens of the various Italian states, including the Papacy, were responsible for substantial improvements in cryptographic practice (eg, polyalphabetic cyphers invented by Leon Alberti[?] ca 1465). And in the Arab world, religiously motivated textual analysis of the Koran led to the invention of the frequency analysis technique for breaking monoalphabetic substitution cyphers sometime around 1000 CE.

Both cryptography, cryptanalysis, and secret agent betrayal featured in the Babington plot during the reign of Queen Elizabeth I which led to the execution of Mary, Queen of Scots. And an encrypted message from the time of the Man in the Iron Mask (decrypted around 1900 by Étienne Bazeries[?]) has shed some, regrettably non-definitive, light on the identity of that legendary, and unfortunate, prisoner. Cryptography, and its misuse, was involved in the plotting which led to the execution of Mata Hari and even more reprehensibly in the travesty which led to Dreyfus' conviction and imprisonment, both in the early 20th century. Fortunately, cryptographers were also involved in setting Dreyfus free; Mata Hari, in contrast, was shot.

Mathematical cryptography leaped ahead (mostly secretly) after World War I. Marian Rejewski, in Poland, attacked and 'broke' the early German Army Enigma system (an electromechanical rotor cypher machine) using theoretical mathematics in (1932. The break continued up to '39, when changes in the way the German Army's Enigma machines were used required more resources than the Poles could deploy. His work was extended by Alan Turing, Gordon Welchman, and others at Bletchley Park beginning in 1939, leading to sustained breaks into several other of the Enigma variants and the assorted networks for which they were used. US Navy cryptographers (with cooperation from British and Dutch cryptographers after 1940) broke into several Japanese Navy[?] crypto systems. The break into one of them famously led to the US victory in the Battle of Midway. A US Army group, the SIS, managed to break the highest security Japanese diplomatic cypher system (an electromechanical 'stepping switch' machine called Purple by the Americans) before WWII began. The Americans referred to the intelligence resulting from cryptanalysis, perhaps especially that from the Purple machine, as 'Magic'. The British eventually settled on 'Ultra' for intelligence resulting from cryptanalysis, particularly that from message traffic encyphered by the various Enigmas. An earlier British term for Ultra had been 'Boniface'.

World War II Cryptography

By World War II mechanical and electromechanical cryptographic cypher machines were in wide use, although where these were impractical manual systems continued to be used. Great advances were made in both practical and mathematical cryptography in this period, all in secrecy. Information about this period has begun to be declassified in recent years as the official 50-year (British) secrecy period has come to an end, as the relevant US archives have slowly opened, and as assorted memoirs and articles have been published.

The Germans made heavy use (in several variants) of an electromechanical rotor based cypher system known as Enigma, the Japanese Foreign Office used an independently developed electrical stepping switch based system (called Purple by the US), and also used several similar machines for attaches in some Japanese embassies. One of these was called the 'M-machine' by the US, another was referred to as 'Red'. All were broken, to one degree or another by the Allies. The German military also deployed several mechanical implementations of one-time pads[?]. Bletchley Park called them the Fish cyphers[?], and Max Newman and colleagues designed and deployed the world's first programmable electronic computer, the Colossus, to help with those cypher systems.

Other cypher machines used in WWII included the British Type X and the American SIGABA; both were electromechanical rotor designs similar in spirit to the Enigma. Neither is known to have been broken by anyone during the war.

Modern Cryptography

The era of modern cryptography really begins with Claude Shannon, arguably the father of mathematical cryptography. In 1949 he published the paper Communication Theory of Secrecy Systems (http://www3.edgenet.net/dcowley/docs) in the Bell System Technical Journal and a little later the book, Mathematical Theory of Communication, with Warren Weaver. These, in addition to his other works on information and communication theory established a solid theoretical basis for cryptography and for cryptanalysis. And with that, cryptography more or less disappeared into secret government communicatiosn organisations such as the NSA. Very little work was again made public until the mid '70s, when everything changed.

1976 saw two major public (ie, non-secret!) advances. First was the DES (Data Encryption Standard) submitted by IBM, at the invitation of the National Bureau of Standards (now NIST), in an effort to develop secure electronic communication facilities for businesses such as banks and other large financial organizations. After 'advice' and modification by the NSA, it was adopted and published as a FIPS Publication (Federal Information Processing Standard) in 1977 (currently at FIPS 46-3). It has been made effectively obsolete by the adoption in 2001 of the Advanced Encryption Standard, also a NIST competition, as FIPS 197. DES was the first publicly accessible cypher algorithm to be 'blessed' by a national crypto agency such as NSA. The release of its design details by NBS stimulated an explosion of public and academic interest in cryptography. DES and more secure variants of it (such as 3DES, see FIPS 46-3) are still used today, although DES was officially supplanted by AES (Advanced Encryption Standard) in 2001 when NIST announced the selection of Rinjdael, by two Belgian cryptographers, as the AES. DES remains in wide use nonetheless, having been incorporated into many national and organizational standards. However, it has been broken (by the Electronic Frontier Foundation, a cyber civil rights group -- the story is in Cracking DES, published by O'Reilly and Associates) -- and it should not be used in new crypto system designs.

Second, and perhaps even more important, was the publication of the paper New Directions in Cryptography (http://citeseer.nj.nec.com/340126) by Whitfield Diffie[?] and Martin Hellman[?]. This paper introduced a radically new method of distributing cryptographic keys, which went far toward solving one of the fundamental problems of cryptography, key distribution. It has become known as Diffie-Hellman key exchange[?]. The article also seems to have stimulated the almost immediate public development of a new class of encyphering algorithms, the asymmetric key algorithms[?].

Prior to that time, all useful modern encryption algorithms had been symmetric key algorithms, in which the same cryptographic key is used with the underlying algorithm by both the sender and the recipient who must both keep it secret. All of the electromechanical machines used in WWII were of this logical class, as were the Caesar and Atbash cyphers and essentially all cypher and code systems throughout history. The 'key' for a code is, of course, the codebook, which must likewise be distributed and kept secret.

Of necessity, a key in every such system had to be exchanged between the communicating parties in some secure way prior to any use of the system (the term usually used is 'via a secure channel') such as a trustworthy courier with a briefcase handcuffed to a wrist, or face-to-face contact, or a loyal carrier pigeon. This requirement rapidly becomes unmanageable when the number of participants increases beyond some small number, or when (really) secure channels aren't available for key exchange. In particular, a separate key is required for each communicating pair if other parties are not to be able to decrypt their messages. A system of this kind is also known as a private key, secret key, or conventional key[?] cryptosystem. D-H key exchange (and succeeding improvements) made operation of these systems much easier, and more secure, than had ever been possible before.

In contrast, in asymmetric key encryption, there is a pair of mathematically related keys for the algorithm, one of which is used for encryption and the other for decryption. Some, but not all, of these algorithms have the additional property that one of the keys may be made public since the other cannot be (by any currently known method) deduced from the 'public' key. The other key in these systems is kept secret and is usually called the 'private' key. An algorithm of this kind is known as a public key / private key algorithm[?], although the term asymmetric key cryptography is preferred by those who wish to avoid the ambiguity of using that term for all such algorithms, and to stress that there are two distinct keys with different secrecy requirements.

As a result, only one key pair is now needed per recipient (regardless of the number of senders) as possession of a public key (by anyone whatsoever) does not compromise the 'security' of the algorithm so long as the corresponding private key is not known to any attacker (effectively this means not known to anyone except the sender). These algorithms made practical, and possible, the widespread deployment of high quality crypto systems which could be used by anyone. This gave government crypto organizations worldwide a severe case of heartburn; for the first time, those outside that fraternity might have access to cryptography that wasn't readily breakable by the snooping side of those organizations. Considerable controversy, and conflict, began almost immediately. It has not yet died down. (See S Levy's Crypto for a journalist's account of the policy controversy in the US).

Note, however, that it has NOT been proven, for any of the good public/private asymmetric key algorithms, that a private key cannot be deduced from a public key (or vice versa). However, informed observers believe it to be currently impossible (and perhaps forever impossible) for the 'good' asymmetric algorithms; no workable deduction techniques have been publicly shown for any of them. Note also that some asymmetric key algorithms have been quite thoroughly broken, just as many symmetric key algorithms have; there is no special magic attached to using two keys.

In fact, some of the well respected, and most widely used, public key / private key algorithms can be broken by one or another cryptanalytic attack and so, like most encryption algorithms, the protocols within which they are used must be chosen and implemented carefully. _All_ of them can be broken if the key length used is short enough to permit practical brute force key search; indeed this is true of all encryption algorithms using keys, regardless of their type.

This is an example of the fundamental problem for those who wish to keep their communications secure; they must choose a crypto system (algorithms + protocols + operation) that resists all attack from any attacker. There being no way to know who those attackers might be, nor what resources they might be able to deploy, nor what advances in cryptanalysis (or its associated mathematics) might in future occur, users may ONLY do the best they know how, and then hope. In practice, for well designed / implemented / used crypto systems, this is believed by informed observers to be enough, and possibly even enough for all(?) future attackers. Distinguishing between well designed / implemented / used crypto systems and crypto trash is another, quite difficult, problem for those who are not themselves expert cryptographers. It is even quite difficult for those who are.

non-secret encryption

Asymmetric key cryptography, D-H key exchange, and the best known of the public key / private key algorithms (ie, what is usually called the RSA algorithm), all seem to have been developed at a UK intelligence agency before the public announcement by Diffie and Hellman in '76. GCHQ has released documents claiming that they had developed public key cryptography before the publication of Diffie and Hellman's paper. Various classified papers were written at GCHQ during the 1960s and 1970s which eventually led to schemes essentially identical to RSA encryption and to Diffie-Hellman key exchange in 1973 and 1974. Some of these have now been published, and the inventors (James Ellis, Clifford Cocks, and Malcolm Williamson) have made public (some of) their work.

Some algorithms of various kinds

Hash functions, aka message digest functions, cryptographic hash functions

• MD5
• SHA-1
• RIPEMD-160
• Tiger

open source crypto systems (algorithms + protocols + system design)

• PGP
• GPG
• SSH
• IPSec / Free S/WAN[?]

Public key / private key encryption algorithms (aka asymmetric key algorithms)

• Diffie-Hellman
• El Gamal[?]
• Elliptic curve cryptography
• RSA

Secret key algorithms (aka symmetric key algorithms)

• Enigma (WWII German rotor cypher machine -- many variants, many users)
• Purple (WWII Japanese Foreign Office cypher machine)
• SIGABA (WWII US cypher machine)
• JN-25[?] (WWII Japanese Navy superencyphered code)
• One-time pad (Vernam and Mauborgne, patented mid-'20s)
• Data Encryption Standard (DES, FIPS 46-3, 1976)
• Lucifer cipher[?] (IBM, early 1970s; modified to become DES)
• RC4 cipher (one of a series by Ron Rivest of MIT)
• Blowfish (by Bruce Schneier, et al)
• International Data Encryption Algorithm (IDEA -- J Massey and X Lai)
• Advanced Encryption Standard (AES, FIPS 197, 2001 -- by Daemen and Rijmen)
• Twofish (AES finalist, by Schneier et al)
• RC6 (AES finalist, by Rivest et al)
• MARS[?] (AES finalist, by Don Coppersmith et al)
• Serpent (AES finalist, by Ross Anderson et al)
• Iraqi Block Cipher (IBC)

Pseudo-random number generators

• Blum Blum Shub
• Yarrow (by Schneier, et al)
• Fortuna (by Schneieir, et al)
• ISAAC

Anonymous communication

• Dining cryptographers protocol (by David Chaum)
• Anonymous remailer

Terminology

• Cryptographic key
• Cipher
• Code
• Brute force attack
• Dictionary attack
• Unicity distance

Further Reading

• General note on cryptographic references: There is a great amount of myth and misunderstanding in wide circulation about topics cryptographic. Some is grossly wrong, some is 'merely' subtly misleading, much of it is plausible to the crypto newcomer and even to the somewhat experienced. There is also a very great selection of poorly done, non-secure crytographic software on the market (purchaseware, shareware, freeware, journalware, xyzware). Readers, buyers, and users should exercise _substantially_ more than the usual caution lest they lose one, two, or all of the reasons they have bothered with cryptography at all (see the article above for the goals of cryptography). At the time this sentence was written, each of the following references is reliable. Mostly. Consider that none covers up_to_date secret government cryptography (at minimum, publishing schedules do not permit it, more generally NSA and brethern don't talk), none is even complete for material available before publication, and none is error free. All of this, plus individual differences in comprehension of a complex field, may produce considerable distortions in your understanding of the current state of the art in cryptography. Nevertheless, try these references first if you wish to minimize those distortions.
• The Beginner's Guide to Cryptography (http://www.murky.org/cryptography/index.shtml) - This website gives a (quite) elementary overview of a few basic areas of cryptography.
• Ferguson, Niels, and Schneier, Bruce - Practical Cryptography, Wiley, 2003, ISBN 0471223573. Up to date cryptography reference. Covers both algorithms and protocols. This is an in depth consideration of one cryptographic problem, including paths not taken and some reasons why. Most of the material is not otherwise available in a single source. Some is not otherwise available. In a sense, a follow-up to 'Applied Cryptography'.
• Schneier, Bruce - Applied Cryptography, 2 ed, Wiley, ISBN 0471117099. The best single volume available covering modern cryptographic practice and possibilities. About as comprehensive as a single volume could have been. Not overly mathematical, well written, and so accessible -- mostly -- to the non-technical.
• Schneier, Bruce - Secrets and Lies, Wiley, ISBN 0471253111, a discussion of the context within which cryptography and cryptosystems work. Meta-cryptography, if you will. Required reading for would be cryptographers, and nearly so for all cryptography users.
• Ross Anderson -- Security Engineering, advanced coverage of computer security issues, including cryptography, by one of its foremost practicioners, and most likely its best writer.
• Bamford, James - The Puzzle Palace : A Report on America's Most Secret Agency ISBN 0140067485, and the more recent "Body of Secrets". The best of a quite small group of books about NSA. Most are inadequate, and untrustworthy, for various reasons.
• A. J. Menezes, P. C. van Oorschot and S. A. Vanstone - Handbook of Applied Cryptography ISBN 0849385237 (online version (http://cacr.math.uwaterloo.ca/hac/)). Equivalent to Applied Cryptography in many ways, but seriously mathematical.
• Kahn, David - The Codebreakers ISBN 0684831309 The best available single volume source for cryptographic history, at least for events up to the mid '60s. The added chapter on more recent developments (in the most recent edition) is regrettably far too thin. See also his other publications on cryptography which have been uniformly excellent.
• Piper, Fred and Sean Murphy - Cryptography : A Very Short Introduction ISBN 0192803158 This book quickly sketches out the major goals, uses, methods, and developments in cryptography.
• Singh, Simon - The Code Book ISBN 1857028899. An anecdotal introduction to the history of cryptography, but much better than such an approach might be expected to produce. Covers more recent material than does Kahn's The Codebreakers. Well written. Sadly, the included cryptanalytic contest has been won and the prize awarded; the cyphers are still worth having a go at, however.

Related topics Echelon, Enigma, Espionage, IACR, Purple code, Ultra, Security engineering, SIGINT, Steganography, Cryptographers, SSL, Quantum Cryptography, Crypto-anarchism, Cypherpunk