How to use the Entropy and Information Gain Calculator
- 1. Enter the parent countsType how many rows of each class sit in the node, like 9, 5.
- 2. Enter each childOne line per branch with counts in the same class order.
- 3. CompareSee entropy, information gain, gain ratio and Gini gain.
Worked example
A node has 9 Yes and 5 No. Its entropy is −(9/14)log₂(9/14) − (5/14)log₂(5/14) = 0.9403 bits. A split by Outlook makes three children: Sunny (2 Yes, 3 No), Overcast (4, 0) and Rain (3, 2). Their entropies are 0.971, 0 and 0.971, and the size-weighted average is (5 × 0.971 + 4 × 0 + 5 × 0.971) ÷ 14 = 0.6935. Information gain is 0.9403 − 0.6935 = 0.2467.
Gain ratio and Gini
Information gain favours features with many values; an ID column would split every row into its own pure leaf and look perfect. C4.5 fixes this with gain ratio = gain ÷ split information, where split information is the entropy of the branch sizes themselves. For the example, the split information is 1.5774, so the gain ratio is 0.1564.
- Entropy is 0 for a pure node and 1 bit for a 50/50 two-class node.
- Gini impurity = 1 − Σ p². For 9/5 it is 0.4592, and the Outlook split's Gini gain is 0.1163.
- For more than two classes, entropy can exceed 1 bit: four equal classes give 2 bits.
Every count must be zero or more, and the children must add up to the parent. To let the computer try every feature for you, use the decision tree visualizer.
Frequently asked questions
Why log base 2?
It measures entropy in bits, the usual convention for decision trees. Natural log gives nats; the best split is the same either way.
Can I use more than two classes?
Yes. Enter as many counts per line as you have classes, in the same order for parent and children.
Is the Entropy and Information Gain Calculator free?
Yes, the Entropy and Information Gain Calculator is free. The first time you see your first result, you are asked for your name and WhatsApp number once. After that, this and every other Uncodemy tool opens straight away without the form.