Python Character Set and Tokens (CBSE Class 11 Computer Science)

Python programs are made up of characters and tokens. Just as English sentences are formed using alphabets, words, and punctuation marks, Python programs are written using a predefined set of characters and tokens. Understanding these basic building blocks is essential before learning Python programming.

Learning Objectives

• Understand the Python Character Set.
• Learn Python Tokens.
• Understand Keywords.
• Understand Identifiers.
• Learn Literals.
• Learn Operators.
• Understand Punctuators.

---

Python Character Set

A Character Set is the collection of valid characters that can be used to write Python programs.

---

Characters Used in Python

Category	Examples
Letters	A-Z, a-z
Digits	0-9
Whitespace	Space, Tab, New Line
Special Symbols	+, -, *, /, %, =, (), [], {}, :, ;, ., ,, _, #, @

---

Python Tokens

A Token is the smallest meaningful unit of a Python program.

Python Tokens

      │

      ├── Keywords

      ├── Identifiers

      ├── Literals

      ├── Operators

      └── Punctuators

---

1. Keywords

Keywords are reserved words that have predefined meanings in Python. They cannot be used as variable names or identifiers.

Examples

if

else

while

for

break

continue

True

False

None

def

class

return

import

---

Example

if marks >= 33:

    print("Pass")

Here, if is a keyword.

---

Important Points

• Keywords are reserved words.
• Keywords have predefined meanings.
• Keywords cannot be redefined.
• Keywords cannot be used as variable names.

---

2. Identifiers

An Identifier is the name given to variables, functions, classes, modules, and other objects in Python.

Examples

studentName

total_marks

salary

radius

calculateArea

---

Rules for Naming Identifiers

• Must begin with a letter or underscore (_).
• Cannot begin with a digit.
• Can contain letters, digits and underscores.
• Cannot contain spaces.
• Cannot use Python keywords.
• Python is case-sensitive.

---

Valid Identifiers

Identifier	Valid?
student	✔ Yes
roll_no	✔ Yes
_marks	✔ Yes
salary2025	✔ Yes

---

Invalid Identifiers

Identifier	Reason
2marks	Starts with digit
my marks	Contains space
for	Keyword
student-name	Contains hyphen

---

3. Literals

A Literal is a fixed value written directly in a program.

Examples

Literal Type	Example
Integer	100
Float	45.67
String	"Python"
Boolean	True
None	None

---

Example

age = 18

name = "Rahul"

passed = True

Here, 18, "Rahul" and True are literals.

---

4. Operators

Operators perform operations on variables and values.

Examples

Operator	Purpose
+	Addition
-	Subtraction
*	Multiplication
/	Division
%	Modulus
==	Comparison
and	Logical AND
or	Logical OR

---

Example

a = 20

b = 10

c = a + b

---

5. Punctuators

Punctuators are special symbols that organize and structure Python programs.

Common Punctuators

Punctuator	Purpose
()	Function calls
[]	Lists
{}	Dictionaries
:	Start of block
,	Separate items
.	Access methods and attributes
' ' or " "	String delimiters

---

Example

numbers = [10,20,30]

print(numbers)

---

Think Like a Programmer

Identify the tokens in the following program:

name = "Aman"

age = 17

if age >= 18:

    print("Eligible")

Token	Type
name	Identifier
"Aman"	Literal
=	Operator
if	Keyword
()	Punctuator

---

Real-Life Example

Think of writing an English sentence:

• Letters form words.
• Words form sentences.
• Punctuation marks improve readability.

Similarly, Python programs are built using characters, tokens, and punctuators.

---

Common Errors

• Using keywords as variable names.
• Starting identifiers with digits.
• Using spaces in identifiers.
• Confusing literals with identifiers.
• Incorrect use of punctuators.

---

Exam Tips

---

Frequently Asked Questions (FAQs)

1. What is a Python token?

A token is the smallest meaningful unit of a Python program.

2. Can keywords be used as variable names?

No. Keywords are reserved words and cannot be used as variable names.

3. What is an identifier?

An identifier is the name given to variables, functions, classes, and other objects.

4. What is a literal?

A literal is a fixed value written directly in a Python program.

5. What are punctuators?

Punctuators are special symbols used to organize and structure Python programs.

---

Summary

• The Python Character Set includes letters, digits, whitespace, and special symbols.
• Tokens are the basic building blocks of a Python program.
• The five types of Python tokens are Keywords, Identifiers, Literals, Operators, and Punctuators.
• Keywords are reserved words with predefined meanings.
• Identifiers follow specific naming rules.
• Literals represent fixed values.
• Operators perform operations, while punctuators help structure Python code.