Using multiple algorithms, ensemble learning, with python implementation

Photo by Cliff Johnson on Unsplash

People consult other people’s opinions before making decisions on most issues. While deciding a collective environment, the decision is usually made when the majority says. While this is the case even at the individual level, some companies survey many things at the global level. Decisions made by a collective crowd, not by a single expert, are called “wisdom of the crowd” and Aristotle used this argument for the first time in his work named Politics. The part where this approach is integrated into machine learning is ensemble learning. In its simplest form, it is about training multiple models and comparing their results to solve complex problems. The aim is to increase the accuracy rate under surveillance by considering generalization and to achieve more successful results. This article involves the decision mechanism by minimizing methods and how they are implemented in python.

Voting classifier, as the name suggests, is a ‘vote’ -democracy-based classification. To explain in a single sentence, it can be defined as majority voting. Let’s assume that we train our classification problem with 3 different algorithms — SVM, LogisticRegression, Decision Trees — and we achieve different success rates in each. Voting Classifier, on the other hand, evaluates the result of each test data specifically and decides on the side with the majority of votes. The test results of the trained data are given in Figure 1, and incorrect predictions are compared with y_test and shown in red. Looking at the Accuracy results, huge differences can be seen. Of course, it is not expected that there will be such a difference in the real dataset, but Figure 1. illustrates how the voting classifier works. Since there are 3 classifiers, the voting classifier has decided on the side where at least 2 of them are the same. This classification, which is democracy-majority vote-based, is called the hard voting classifier.

Figure 1. How does Voting Classifier work?, Image by author

Soft voting classifier works with a more flexible policy. Just like in hard voting, the result of each classifier is evaluated, but one step back comes to the fore, that is, it observes the probability values of…

Continue reading:—-7f60cf5620c9—4