← CSC 315 Artificial Intelligence
Machine Learning and Decision Tree Induction
Notes
Transition from Symbolic AI to Learning Agents
While classical symbolic artificial intelligence relies on explicit human knowledge engineering, modern learning agents acquire decision policies directly from empirical datasets. Machine learning equips agents with inductive reasoning capabilities, enabling them to discover general underlying patterns from historical observations. In supervised learning paradigms, an agent receives a training set comprising labeled input-output pairs and learns a mapping hypothesis function capable of generalizing accurately to unseen test instances. Classification tasks map continuous or categorical feature vectors into discrete categorical class labels.
Decision Tree Learning and Information Gain
Decision trees are popular symbolic classification models that recursively partition the feature space into orthogonal subregions. Each internal node represents an evaluation of a specific feature, each branch signifies the outcome of that evaluation, and each leaf node denotes a terminal class assignment. The construction of an optimal decision tree relies on greedy heuristic split algorithms, such as ID3 or C4.5. These algorithms select the best splitting feature at each node by measuring Information Gain, which quantifies the expected reduction in entropy. Entropy measures the statistical impurity or randomness of a dataset. By choosing splits that maximize Information Gain, the tree achieves maximum class separation using the shallowest possible depth.
Practical example
Consider a mobile fintech credit scoring system developed for a digital lending platform operating across Kenya to evaluate micro-loan default risks on mobile wallets.
The agent analyzes historical borrower records. Each sample contains four attributes: daily average M-Pesa transaction volume, airtime purchase frequency, business license registration status, and late utility bill occurrences. The target classification label is credit risk status, categorized as either Low Risk or High Risk.
At the root node, the dataset contains 1,000 applicants with an initial entropy indicating substantial class mixing. The algorithm calculates the Information Gain for each attribute. M-Pesa transaction volume yields an Information Gain of 0.42, airtime frequency yields 0.15, business license registration yields 0.28, and utility payments yield 0.21. The algorithm selects M-Pesa transaction volume as the root decision node.
Applicants transacting over five thousand Kenya Shillings daily branch to an internal sub-node where utility bill consistency is evaluated. Those with no delinquent payments land on a pure leaf node classified as Low Risk, receiving automated micro-loan disbursement. Applicants falling below the transaction threshold with late utility payments are assigned to High Risk, prompting collateral verification or reduced credit limits.
Quiz
1. What type of reasoning does a machine learning agent primarily use when deriving general rules from specific historical data?
2. Why did the credit scoring agent select M-Pesa transaction volume as the root node of the decision tree?
3. In decision tree induction algorithms like ID3, what does entropy measure?
Sign in to submit and securely grade your quiz.