Related classification algorithms
Traditional machine learning models
Traditional
Programming is running data and programs on a computer to produce output and
Machine Learning is running data and output on a computer to create a program.
The program can be used in traditional programming. Machine learning is a science
that allows computers to act without explicit programming[25]. In the past
decade, machine learning has brought us self-driving cars, useful speech
recognition, efficient web search, and a vast understanding of the human
genome. Machine learning is so ubiquitous today that you might be using it
dozens of times a day without even knowing it. In machine learning, there are a
large number of linear classifiers, nonlinear classifiers, statistics-based
classifiers, and probabilistic classifiers, etc. These classification
algorithms usually have the characteristics of fast training speed and clear
action mechanism[37]. This section mainly introduces the Decision Tree,
Gradient Boosting Decision Tree (GBDT), Random Forest, Logistic regression,
SVM, and K-Nearest Neighbor (KNN) algorithm.
(1)Decision
tree
The
decision tree is one of the simplest and most useful machines learning
structures. Decision trees are an approach to machine learning. Decision tree
generation algorithms include ID3, C4.5, and C5.0. A decision tree is a
tree-shaped structure in which each internal node represents a judgment on an
attribute, each branch represents the output of a judgment result, and finally,
each leaf node represents a classification result[38]. Decision tree learning methods
are commonly used in data monitoring. The model is targeted at predicting the
quality of a target available based on input available. Use decision trees to
classify data and build models that describe important class variables for
machine learning and pattern recognition. The output of the classification
problem is treated as a pattern of all observed values of the terminal
node[39]. The method of establishing classification models is mainly divided
into two steps. First, the classification model is established based on
training data. Second, the accurateness of the model is checked and classified
by the new data model. The category labels shown here take the form of discrete
values, such as "Yes" or "No", "Dangerous" or "Safe",
etc.
(2)Gradient
lifting tree
Gradient
Boosting is a method of boosting. Its main idea is that every time a model is
established, the gradient descent direction of the model loss function is
established before. Gradient boosting can present a more efficient algorithm
and a training dataset faster. It can benefit from regularization methods and
advises different parts of the algorithm and generally improves the
effectiveness of the algorithm by reducing overfitting. The function of the
loss is that the performance of the model is believed that the lower the amount
of damage, the better the efficiency[40]. The loss function used in the
gradient boosting algorithm may vary depending on the type of problem being
solved, but many can support the standard loss function and define its own
functions. For example, use the log decay function that can use the regression
square loss difference function and classification.
The
GBDT algorithm uses the decision tree as weak learning of the gradient boosting
model[6][41]. First, each weak student will weigh. The program will run a
specific set of training images. Then, given the results, it will re-weigh the
array of weaker students. If a much better estimate is made than before, the
effect will be raised accordingly. GBDT is a greedy algorithm that can quickly
fit training data sets. This allows the output of subsequent models to be added
and residuals in the prediction to be "corrected". For example, the
classification of binary images. The decision tree is constructed in a greedy
way and the best split point is selected according to the Gini index. GBDT
algorithm can use the "ensemble" method of gradient boosting in
almost any type of ML project, from HSI recognition to the classification of
user recommendations or natural language analysis.
(3)Random
forests
A
random forest is a decision tree that is generated using a set of the large
tree and mathematical algorithms that give equal probabilities of trees in a
given distribution. The random tree is most commonly generated with a random
tree generator. Each tree produces category predictions, and the category with
the most votes become the prediction of the model[42]. Two methods, such as
bagging and feature random selection are used randomly to ensure that each
individual tree in the random forest does not have too much to do with the
behavior of another tree.
Random
forest algorithm creates decision trees on data samples and then gets the
prediction from each of them and finally selects the best solution employing
voting. The performance of decision trees is easily unstable due to the quality
of samples. Random forest allows each individual tree to be replaced by random
sampling from the data set. Similarly, to obtain different decision trees in
this way, this process is called bagging. They reduce errors using greedy
algorithms that want to split variables[42]. For example, decision trees can
have many structural similarities with bagging and instead have a high
correlation in their predictions. It should be noted that the use of bagging
does not divide the training data into multiple parts, nor does it train each
tree into different data blocks. In the models in this series, the difference
between the decision trees is even greater, and ultimately the higher the
correlation between the decision trees. As the Bagged decision trees are
constructed and can calculate how much the error function drops for a variable
at each split point. Therefore, the final decision tree in random forests not
only needs to be trained for different data sets later, but it also needs to
use different functions for decision making.
(4)Logistic
regression
In
statistics, logical models are usually used to model the probability of a
specific category or event. Logistic regression is a machine learning method
for solving binary (0 or 1) problems, which is used to estimate the probability
of something[39]. Logical regression assumes that the dependent variable
follows a Bernoulli distribution. Logistic regression easily introduces
nonlinear components through the sigmoid function, so it can easily deal with
0/1 classification problems. Logical regression assumes that the dependent
variable follows a Bernoulli distribution, while linear regression assumes that
the dependent variable follows a Gaussian distribution. First of all, need to
introduce the Sigmoid function, also known as the Logistic function. Similar to
linear regression, which assumes that the data follows the linear function,
logistic regression also uses the S-shaped function to model the data, as
follows:
Logical
regression can be used as a classifier only when a decision threshold is
introduced. The setting of the threshold is an important element of logistic
regression and depends on the classification problem itself.
(5)Support vector machine
Support
vector machine is a supervised machine learning algorithm. It is a two-class
classification model. Its basic model is defined as the linear classifier with
the largest interval in the feature space. Its learning strategy is to maximize
the interval, which can finally be transformed into the solution of a convex
quadratic programming problem. The basic idea is to find the farthest boundary
in each category and determine whether the sample belongs to a region or other
region by solving the basic mathematical function given the coordinates of each
sample. In addition to linear classification, support vector machines can use
so-called nuclear techniques, and their inputs are implicitly mapped into
high-dimensional feature spaces for effective nonlinear classification. The SVM
algorithm is implemented using the kernel, and the hyperplane learning in
linear SVM is accomplished by using some linear alleged conversion
problems[43].

Among
the traditional machine learning algorithms, SVM has some irreplaceable
advantages. For example, SVM learns faster and performs better when the number
of samples is limited, making the algorithm well suited for text classification
problems, so for such cases, up to thousands of data sets with labeled samples
can usually be accessed
(6)K
- nearest neighbor
The
k-nearest neighbors (KNN) algorithm is a simple, easy-to-implement supervised
machine learning algorithm that can be used to solve both classification and
regression problems. The KNN algorithm belongs to the overall scope of
supervised learning, and its classification principle is based on the simple
thing that is accepted as true or as certain to happen, without proof that
similar things are always closer. A method in K-mean clustering vector
quantization and the K-means algorithm is used to cluster to determine and can
be used for Regression as well as for classification but mostly it is used for
the classification problems, it is a non-parametric algorithm, which means it does
not make any assumption on underlying data [44]. The K-NN algorithm estimates
seminaries between new case and data and available case and keeps the new case
and data in categories that are similar to the available categories. K-NN
algorithm stores all the available data and classifies a new data point based
on the similarity, in other words, KNN algorithm at the training phase just
stores the dataset and when it gets new data, then it classifies that data into
a category that is much similar to the new data.
The
KNN algorithm has the characteristics of being very easy to explain and
understand, and despite its simplicity, it can still produce highly competitive
results. At the same time, KNN algorithm is a non-parametric algorithm, which
does not need to make any assumptions about data distribution, which makes KNN
more advantageous in processing special data. Higher-dimensional data can cause
the accuracy of the KNN algorithm to decline because the nearest element is
almost indistinguishable from the farthest element.

0 মন্তব্য(গুলি):
একটি মন্তব্য পোস্ট করুন