Flex (Fast Lexical Analyzer Generator)

Flex (Fast Lexical Analyzer Generator), or simply Flex, is a tool for generating lexical analyzers scanners or lexers. Written by Vern Paxson in C, circa 1987,

• Flex is designed to produce lexical analyzers that is faster than the original Lex program.
• It is often used along with Berkeley Yacc or GNU Bison parser generators. Both Flex and Bison are more flexible, and produce faster code, than their ancestors Lex and Yacc.

Role of Flex in Compiler Design

Flex takes a specification file (with extension .l) containing regular expressions and actions, and automatically generates a C program that performs lexical analysis.

• Flex generates a function called yylex()
• yylex() reads the input stream character by character
• It identifies tokens based on the given rules
• These tokens are then passed to the parser

Installing Flex on Ubuntu:

sudo apt-get update
sudo apt-get install flex

Note: If the update command has not been executed on the system for a long time, it is recommended to run it first. This ensures that the latest package versions are installed, since older versions may be incompatible with newly installed packages or may no longer be available in the repository.

How Flex Works (Workflow)

1. Lexical specification file lex.l is written using Flex syntax.
2. Flex compiler converts lex.l into a C file named lex.yy.c.
3. C compiler compiles lex.yy.c into an executable (usually a.out).
4. Generated executable reads the input characters and converts them into a sequence of tokens.

Structure of a Flex Program

1. Definition Section: This section contains:

• Variable declarations
• Header files
• Macro definitions

Anything written inside %{ %} is copied directly into lex.yy.c.

Syntax:

%{
 // Definitions
%}

2. Rules Section: This section defines patterns and actions.

• Each rule has the form: pattern { action }
• Patterns are written using regular expressions
• Actions are written in C code
• The section is enclosed between %%

Syntax:

%%
pattern action
%%

Examples: Table below shows some of the pattern matches.

3. User Code Section: This section contains:

• main() function
• Supporting C functions
• yywrap() definition

Linked with the generated lexical analyzer.

Basic Program Structure:  

%{
// Definitions
%}

%%
Rules
%%

User code section

How to run the program:

First save the file with the extension .l or .lex. After saving the file, execute the following commands in the terminal to compile and run the program.

Step 1: flex filename.l or flex filename.lex depending on the extension file is saved with 
Step 2: gcc lex.yy.c 
Step 3: ./a.out 
Step 4: Provide input to program if it is required

Note: To stop the input, press Ctrl + D (EOF) in the terminal, or define a rule in the program to terminate input. You can refer to the output examples of the programs below if you face difficulty while running them.

Example 1: Count the number of characters in a string

If the user enters:

GFG123gfg

Output:

G capital letter
F capital letter
G capital letter
1 not capital letter
2 not capital letter
3 not capital letter
g not capital letter
f not capital letter
g not capital letter

Number of capital letters in the given input - 3

Example 2: Count the number of characters and number of lines in the input

Input Given:

Geeks
for
Geeks
end

Output:

number of lines = 3, number of chars = 13

Advantages

• Efficiency: Flex-generated lexical analyzers are very fast and efficient, which can improve the overall performance of the programming language.
• Portability: Flex is available on many different platforms, making it easy to use on a wide range of systems.
• Flexibility: Flex is very flexible and can be customized to support many different types of programming languages and input formats.
• Easy to Use: Flex is relatively easy to learn and use, especially for programmers with experience in regular expressions.