Unicode - UTF-8, UTF-16 and UTF32

Unicode is a universal character encoding standard designed to represent text and symbols from all writing systems around the world.

• Unicode is the most fundamental and universal character encoding standard. For every character, there is a unique 4 to 6-digit unique hexadecimal number.
• Unicode is standardized among all global computing platforms, devices and programs, enabling consistent representation and manipulation of text across different systems and applications.
• Unicode supports multiple languages, mathematical symbols, emojis and specialized symbols.
• Unicode is flexible. It allows new characters to be added, supporting the evolving communication and language needs.

How is Unicode Compatible with ASCII?

• We can also say that ASCII is a subset of Unicode.
• But wait! For the character 'A', the ASCII representation is 0065 and the unicode point is U+0041. How is it backward compatible with ASCII?
• This is because the U+0041 is in hexadecimal form! which corresponds to 0065 in Decimal.(0041)₁₆ = (0065)₁₀

Size and Growth

Unicode is an extensive and continually evolving character-encoding standard. It currently includes more than 149,000 characters, spanning global writing systems, symbols, and technical notations. As new characters are introduced, the Unicode set continues to expand. Below are a few examples of common characters along with their corresponding Unicode values:

How To Type in Unicode Characters?

• Open your computer and log into your Operating System.
• Open the Unicode character panel. On Windows, press Windows Key + . (period). On macOS, press Control + Command + Space.
• This will open a small window with Unicode characters.
• Search for the character you want and click on it. The character will appear on the screen.

Unicode Transformation Format (UTF)

Unicode Transformation Format is a method of encoding unicode characters for storage and communication purposes. This format specifies how Unicode characters will be converted into a sequence of bytes. The most common UTF forms are UTF-8, UTF-16, UTF-32.

UTF-8

• UTF-8 is a variable width encoding system where each character is encoded into 1 to 4-byte unicode points.
• UTF-8 is backward compatible with ASCII. All the ASCII characters (0-127) and ₁₀ are represented inside UTF-8 (00-F7)₁₆ using one byte.
• Other Unicode characters in UTF-8 are represented using multiple bytes.
• UTF-8 is widely used in internet and UNIX-like operating systems.

UTF-16

• UTF-16 is also a variable width encoding system where each character is encoded into a 2 to 4-byte unicode point.
• UTF-16 is used in Microsoft Windows OS and programming languages like Java

UTF-32

• UTF-32 is a fixed-width encoding system where each character is encoded into 4-byte unicode point.
• This format provides a simple one-to-one correspondence between Unicode characters but makes it less space-efficient, as where it should only take 1 byte of data (Example: 01), it is taking up 4 bytes (Example: 00000001).
• UTF-32 is less commonly used in mainstream applications and systems due to its space inefficiency and compatibility considerations

History of Unicode

There have been numerous versions of Unicode released till now :