Extending Support Vector Regression to Constraint Optimization: Application to the Reduction of Potentially Avoidable Hospitalizations Huu Tu Ngo, Vera Georgescu, Carmen Gervet, Anne Laurent, Thérèse Libourel Rouge, Grégoire Mercier
To cite this version: Huu Tu Ngo, Vera Georgescu, Carmen Gervet, Anne Laurent, Thérèse Libourel Rouge, et al.. Extending Support Vector Regression to Constraint Optimization: Application to the Reduction of Potentially Avoidable Hospitalizations. SoGood - ECML PKDD Workshops, Sep 2018, Dublin, Ireland. pp.89-102. �lirmm-02054874�
HAL Id: lirmm-02054874 https://hal-lirmm.ccsd.cnrs.fr/lirmm-02054874 Submitted on 7 Oct 2019
HAL is a multi-disciplinary open access archive for the deposit and dissemination of scientific research documents, whether they are published or not. The documents may come from teaching and research institutions in France or abroad, or from public or private research centers.
L’archive ouverte pluridisciplinaire HAL, est destinée au dépôt et à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d’enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.
Extending Support Vector Regression to Constraint Optimization: Application to the Reduction of Potentially Avoidable Hospitalizations Tu Ngo1 , Vera Georgescu1 , Carmen Gervet3 , Anne Laurent2 , Th´er`ese Libourel3 , and Gr´egoire Mercier1 1
Economic Evaluation Unit, University Hospital of Montpellier, France 2 LIRMM, University of Montpellier, France 3 Espace-Dev, University of Montpellier, France
Abstract. It has been identified that reducing potentially avoidable hospitalizations (PAHs) not only enhances patients’ quality of life but also could save substantial costs due to patient treatments. In addition, some recent studies have suggested that increasing the number of nurses in selected geographic areas could lead to the reduction of the rates of potentially avoidable hospitalizations in those areas. In the meantime, health authorities are highly interested in solutions improving health care services to reduce the potentially avoidable hospitalizations. The first approaches could be based on descriptive statistics such as actual rates of potentially avoidable hospitalizations at the geographic area level. These simple approaches have limitations since they do not consider other potential factors associated to the high rates of potentially avoidable hospitalizations. Therefore, in this paper, we propose an approach using support vector machine for regression to select not only the geographic areas but also the number of to-be-added nurses in these areas for the biggest reduction of potentially avoidable hospitalizations. In this approach, besides considering all the potential factors, we also take into account the constraints related to the budget and the equality of health care access. In our work, we specifically apply the approach on the Occitanie, France region and geographic areas mentioned above are the cross-border living areas (fr. Bassins de vie - BVs). As we aim at building a user-friendly decision support system, the results of our work are visualized on spatial maps. Although our work is on a specific region and geographic areas, our approach can be extended at the national level or to other regions or countries. Moreover, in this paper, the other methods for regression are also introduced and evaluated as parts of our work. Keywords. Data Mining, Support Vector Machine, Regression, Spatial Maps, Potentially Avoidable Hospitalizations.
1
Introduction
Potentially avoidable hospitalizations (PAHs) are defined as hospital admissions that could have been prevented [1]. In particular, these hospitalizations are in 1
fact the consequence of the sudden aggravation of a chronic disease (diabetes, heart failure, respiratory failure). These acute episodes could have been prevented with timely and effective treatments and therefore the hospitalizations could have been avoided [6]. Every year, in France, there are more than 700,000 preventable hospitalizations, associated with a cost of several hundred million Euros for the Health Insurance [3, 7]. That means avoiding these hospital admissions not only could enhance quality of live of the patients but also could decrease substantial costs caused by patient treatments [2, 3]. There have been many previous studies on PAHs and the potential factors that could be associated with high rates of PAHs [3–5]. Some of the recent studies in France have revealed that the higher (age-and-sex-standardized) rates of PAHs are linked to higher mortality rates, lower density of acute care beds and ambulatory care nurses, lower median income, and lower education levels [3, 4]. More specifically, these studies suggested that by increasing the number of nurses at some geographic areas, the number of PAHs in these areas could be reduced [3]. On the other hand, typically in France, the public health decision makers can have influence on the factors related to health care such as the density of physicians, nurses, or the density of hospital beds while socioeconomic determinants such as income and education are not actionable inside the health system sector. Specifically, both the national- and regional-level health authorities are highly interested in enhancing the health care services in order to reduce the number of PAHs. In addition, the health system is subject to strong constraints. In particular, they must provide quality care while controlling associated costs and ensuring equality of access to the health care services. The latter states that all patientcitizens must be able to benefit from the care they need, regardless of their geographical and socioeconomic situation. Hence, being able to select geographic areas in order to maximize the impact of an intervention is of high importance. That gives birth to our work which aims at building a decision support system that recommends the optimal actions targeting on the geographic areas while considering the constraints. In particular, the purpose of our work is to find the geographic areas to increase the nurses for the biggest reduction of PAHs while not only integrating socioeconomic constraints such as the available budgets as well as ensuring the equal access to health care but also considering other potential determinants of PAHs. The geographic areas we mention here are the cross-border living areas (fr. Bassins de vie - BVs) which define the geographic areas in which the inhabitants have access to the most common equipment and services including trade, education, health, etc. 1 In our approach, for every BV, we compare the predicted rates of PAHs before and after trying to add new nurses. Our idea is that the BVs that return the biggest reduction of these predicted values after trying to increase the number of nurses could be the best ones for the actual nurse implementation. Since the 1
defined by French National Institute for Statistics and Economic Studies (INSEE)
2
rates of PAHs are the numeric values, so any regression method could be the option for our approach. In this paper, we present the method of support vector machine for regression (SVR) and our approach of applying this method to find the BVs to add new nurses. We also briefly introduce other regression methods and our evaluation for the reason why we select SVR for our implementation. Regarding the dataset of our work, the hospital discharge data and the potential determinants (variables) are aggregated at BVs (n = 201). This data is collected from many sources including the French Ministry of Health, the National Institute for Statistics and Economic Studies, the Regional Health Agency of Occitanie, and French Health Insurance Fund ambulatory care claims database. In particular, the data includes: – The primary care supply and hospital supply data including the densities of general practitioners, nurses, specialists, the densities of acute beds, travel time to the closest emergency department, and acute care hospital and medical group practice – The socioeconomic data such as the median income, the unemployment rates, the proportion of population having an education level equal or above the baccalaureate, the proportion of population living in isolated rural areas, the proportion of workers in the active population. – The epidemiological data such as the rates of age and sex-adjusted all-cause and premature mortality. Because of the availability of the data, we first focus our work and the result in the Occitanie region in France although our approach can be applied at the national level or in other countries.
2
Related Works
As introduced in the previous section, to select the cross-border living areas (fr. Bassins de vie - BVs) to add nurses for the biggest reduction of potentially avoidable hospitalizations (PAHs), our approach is to compare the predicted rates of PAHs before and after trying to add new nurses in those BVs. As these predicted rates are numerical values, any regression method could be a solution to our problem. In this section, we present several regression methods and our evaluation regarding to our work. 2.1
Multilinear regression
Whenever we need a regression method, the first choice is often multilinear regression because of its simplicity. In the multilinear regression, the predicted value yˆ is a straight line presented as below: yˆ = W X + b 3
(1)
in which W −1 and X ∈ Rd while y ∈ R. In our work, these predicted values are the rates of PAHs: ˆ = WX + b P AH
(2)
Moreover, W = (w1 , w2 , w3 ...) are the coefficient values corresponding to the multi-dimension variable X = (x1 , x2 , x3 ...). Hence, we can extent Equation (2) to the one below. ˆ = w1 x1 + w2 x2 + ... + b P AH (3) in which xi is variable of dimension i. For example, x1 stands for the density of the nurses. As we have introduced above, in our work, we compare the predicted PAH ˆ b ) and after (P AH ˆ a ) trying to add new nurses for the biggest values before (P AH reduction of these predicted PAH values. The reduction rate at each BV can be mathematically presented by: ˆ b − P AH ˆ a = (w1 x1 + w2 x2 + ... + b)b − (w1 x1 + w2 x2 + ... + b)a P AH
(4)
As we only make changes on the density of nurses (represented by x1 ), Equation (4) becomes: ˆ b − P AH ˆ a = (w1 x1 )b − (w1 x1 )a = w1 (x1b − x1a ) P AH
(5)
in which the density of nurses or the number of nurses per 10,000 people is computed as: N umber of nurses ∗ 10, 000 (6) x1b = Size of P opulation When we increase some nurses, for example A nurses, we have: x1a =
(N umber of nurses + A) ∗ 10, 000 Size of P opulation
(7)
Apply (6) and (7) into (5), we have: ˆ b − P AH ˆ a = −w1 ∗ P AH
A ∗ 10, 000 Size of P opulation
(8)
ˆ b − P AH ˆ a ) presents the difference between rates of PAHs In addition, (P AH per 1,000 inhabitants. Therefore, the expected number of PAHs to be reduced (ExpectedPAHReduction) is: ˆ b − P AH ˆ a ) ∗ Size of P opulation ExpectedP AHReduction = (P AH 1, 000
(9)
Finally, applying (8) to (9), we have the result: ExpectedP AHReduction = −w1 ∗ 10 ∗ A 4
(10)
Since Equation (10) will be applied for every BV, it indicates that the expected numbers of PAHs to be reduced are the same for every BV when we increase the same number of the nurses. That is definitely not the answer we are looking for. On the other side, it should be noted that we do not compute the ExpectedPAHReduction as the differences between the actual numbers of PAHs before adding nurses and the predicted numbers of PAHs after adding nurses because by with this approach the BVs to be selected for adding nurses are actually the ones at which the differences (or the errors) between the actual values and the predicted values of PAHs before adding nurses are the biggest. That does not give us the right answer to our problem either. 2.2
K-nearest neighbors for regression
One of the other approaches for regression method is K-nearest neighbors. The ˆ idea of this method in our work is that the predicted rates of PAHs (P AH) of a BV can be computed from other BVs where we have the similarity in all attributes such as the densities of nurses or the levels of education. These similarity-in-attributes BVs are called neighbors. For example, the predicted rate of PAHs of a BV equals the average value of rates of PAHs of its 5 (K=5) nearest-neighbor BVs. One effective method to measure the similarity of the BVs is to use euclidean distance for the values of the attributes of the BVs. In our approach, at first we compute the predicted rates of PAHs for all BVs ˆ b . Then for each BV, we try to add before adding nurses. These values are P AH new nurses, if at least one of its neighbors is changed, then we can have the ˆ a . Finally, we select the BVs for new predicted rate of PAHs for that BV, P AH adding more nurses by the biggest reduction of the expected number of PAHs (ExpectedPAHReduction). ˆ b − P AH ˆ a ) ∗ Size of P opulation ExpectedP AHReduction = (P AH 1, 000 At the beginning, this approach looked promising to us, but it actually does not work in our case because of the following limitations: - When the dimension of the variables (the number of the attributes) is high, then the neighbors will not be able to be changed if we just make small change on one dimension (density of nurses in our case) - Also regarding to the dimension of the variables, changing the size of dimension means changing the opportunities for the BVs to change the new predicted ˆ a ). That leads to the unstable results in our work. rates of PAHs (P AH 2.3
Neural networks for regression
Neural networks could be very promising to any problem regardless of classification or regression. Suppose that if we deploy the neural network with only one layer for regression, the predicted values will be in a linear formula: yˆ = W X 5
Fig. 1. Neural network two hidden layers for regression
in which xo equals 1 and wo equals b part if we compare this formula with the multilinear formula we mentioned previously. As explained in the multilinear regression case above, we cannot use the neural network one layer regression to solve our problem. That means we need at least one more hidden layer for our work (Fig. 1). Unfortunately, after trying with different models: more hidden layers, different activate functions at the hidden layers as well as applying different techniques such as L1, L2 regularization or dropout to avoid overfitting, we have failed to get the better results for the predicted rates of PAHs compared with the support vector machine for regression (SVR) method (Table 1). Another negative point of neural networks is that they work like “black boxes” on how a certain output is produced and therefore it is very difficult to explain their outputs to others. Hence, at the time of this paper, we think that the neural networks method is not the right method for our work. 2.4
Support vector machine for regression and evaluation
Support vector machine (SVM) has been applied widely in classification problems, but it can also be used as a regression method (SVR). The method was introduced by Vapnik and his colleagues [8] and has been applied in many fields such as financial forecasting [9]. More specifically, SVR can be applied to solve both linear and non-linear regression problems [11, 12]. As explained in the multilinear regression section, the linear formula does not work in our case. On the other hand, for the non-linear problems, the way the method works is to transfer the original independent variables x into a new coordinate system ϕ(x) so that in the new coordinate system the non-linear problems turn to the linear problems (Fig. 2). In particular, in the new coordinate system, the formula to compute the predicted values yˆ is shown in Equation (11) [9, 11, 12]: yˆ =
N X
(αi − αi∗ )ϕ(xi )ϕ(x) + b
(11)
i=1
In practice, the number of the new dimensions of ϕ(x) is often very high or even infinite. Hence, computing ϕ(x) from x becomes difficult or even unfeasible. 6
Fig. 2. SVR for non-linear cases [12]
Therefore, a technique called kernel trick, K(xi , xj ) = ϕ(xi )ϕ(xj ), is applied to directly compute ϕ(xi )ϕ(x) rather than computing all ϕ(x). Particularly, the following kernel functions are often used: Polynomial: K(xi , xj ) = (xi , xj )d Gaussian Radial Basic Function - RBF: ||xi − xj ||2 − 2σ 2
K(xi , xj ) = exp
!
Related to our work, after testing all the kernel functions, we have found that RBF returns the predicted values that are closest to the actual rates of PAHs. In addition, comparing with the results from the other regression methods presented previously, the predicted values by SVR using RBF are closest to the actual rates of PAHs (Table 1). More specifically, Table 1 presents the performance of the regression methods on our dataset in which we use both root-mean-square error (RMSE) and mean-absolute error (MAE) values for the performance evaluations [13]: v u N u1 X RM SE = t e2 N i=1 i M AE =
N 1 X |ei | N i=1
In both formulas above, ei (i = 1, 2, 3...N) are the errors (differences) between the predicted values from the regression methods and actual (observed) values. In our work, the predicted value of a BV is computed by using all the BVs except 7
that BV as the training dataset. This approach requires us to repeat the training for any BV. Clearly, this approach does not work for big datasets, but it is not our case. Table 1. Performance evaluations of regression methods on our dataset Method
RMSE
MAE
SVR using RBF Multi-linear regression K-nearest neighbors Neural networks
0.98 1.04 1.03 1.13
0.76 0.82 0.80 0.87
Based on this result and the analysis for the possible application of the regression methods in our work mentioned above, we have agreed that the SVR method is the best choice for our work.
3
Extracting BVs for adding nurses
As we mentioned briefly in the introduction, the purpose of our work is to select the cross-border living areas (fr. Bassins de vie - BVs) in Occitanie, France region for adding nurses for the most effective PAHs reduction. In particular, we select these BVs by comparing the predicted rates of PAHs before and after trying to add new nurses in every BV. The BVs to be selected are the ones that return the biggest reduction of these predicted values. Hereafter we present the ideas in details. 3.1
Possible constraints
The first thing we need to consider is that there are some constraints on the number of nurses to be added. The first constraint should be the budget that the health authorities can spend for the health service improvement. This constraint indicates that the total number of nurses to be added in the whole region is limited. Another constraint we must consider is to ensure equal access to health care for the inhabitant living in the region. The later constraint can be defined by (1) the maximum number of to-be-added nurses in each BV; and (2) making sure that in the to-be-selected BVs, the densities of the nurses must not be greater than a given threshold. The latter to make sure that we do not add nurses in the BVs whose densities of nurses are already high. To sum up, we have three possible constraints in our work as below: – The maximum number of nurses in total that can be added into the whole region. We denote this constrain as maxGlobal – The maximum number of nurses that can be added in each BV. We denote this constrain as maxLocal 8
– The maximum density of nurses that can be reached in each BV. We denote this constrain as maxLocalDensity 3.2
Best numbers of to-be-added nurses and the biggest PAH reduction rates
Fig. 3. Process flow to find the biggest reduction rate of PAH per to-be-added nurse and best number of to-be-added nurses in each BV
After defining the constraints, the second step is to find the best number of nurses to be added in each BV. In particular, in this step, at each BV, we try to add nurses one by one until we reach either the defined maxLocal or the maximum density of nurses maxLocalDensity. Each time adding a nurse, we 9
compute the reduction rate of PAHs per added nurse to identify at each BV (1) the biggest reduction rate (denoted bestReductionRate); and (2) the best number of to-be-added nurses (denoted bestNumber). The whole process is described in the Figure 3. In the process flow described in Figure 3, it should be noted that in our work, the PAHs are the standardized rates per 1,000 people so that we need to compute the number of PAHs to be reduced (variable PAH No changed in Fig. 3) after increasing nurses in order to get the reduction rate of PAHs per to-be-added nurse (rate). One important thing to note here is the SVM function (SVM(densityNurse)) that actually the SVR method we mentioned in the previous section. We firstly train SVR model using the dataset of PAHs and its potential determinants, then we can get the predicted rates of PAHs before and after trying to add nurses to the BVs. The final result of this step will return the list of all the BVs with their information of bestReductionRate and bestNumber of to-be-added nurses. 3.3
BVs to be selected
Fig. 4. Process flow to select the BVs for adding more nurses
10
After having the values of bestReductionRate for all the BVs, the task to find BVs for adding new nurses becomes easy. More specifically, the BVs to be selected are the ones whose bestReductionRate are the biggest. However, to avoid the cases that in the BVs to selected, the actual rates of PAHs are already small, we add one more condition to the BVs to be selected that we only select a BV if its actual rate of PAHs is higher than its predicted rate of PAHs (actualPAH ≥ predictPAH in Fig. 4). The process of finding BVs for adding nurses is described in Figure 4. In this process, we firstly order the list of the BVs descendingly by their bestReductionRate (function orderBVsByBestDeduectionRate in Fig.4). After that we select the top first BVs until either we reach the maximum number of to-be-added nurses (maxGlobal) in the whole region or we reach the last BVs in the list (reach the total number of BVs, nbBVs in Fig.4). There is a note in Figure 4 that BV(Attr, index) function returns the value of the attribute (Attr ) of the BV associated with its index. The output of this step is a list of the to-be-selected BVs (selectedBVs in Fig. 4) for adding more nurses and the best number of to-be-added nurses in each BV. There is a point that this algorithm might return the total number of to-be-added nurses little more than the constraint on the maximum number of can-be-added nurses in the whole region (maxGlobal). But this does not cause any problem as we also know how many to-be-added nurses in every BV and the decision makers can decide to either increase budget or adjust the number of to-be-added nurses in the last BV in the selected list.
4
Results and evaluations
As mentioned in the previous section, the output of the algorithm is the list of BVs where nurses should be added and the number of nurses to be added in order to obtain the highest decrease in the number of PAH. For better visualization for the decision makers, we rely on spatial maps. For example, the map below (Figure 5) recommends the BVs to increase nurses (the darker colors indicate stronger recommendation) and the optimal number of nurses to be added (the labels in red) should be added in those BVs for the biggest reduction of PAH according to the corresponding constraints. Now let us compare our approach with two approaches using simple descriptive statistic methods. The first map (Figure 6) indicates top 15 BVs recommended by the actual rates of PAHs with the condition on the densities of nurses. Specifically, the BVs recommended are the ones whose the actual rates of PAHs are the biggest with the condition that the densities of nurses are smaller than 25 nurses per 10,000 inhabitants. Similarity, the other map (Figure 7) indicates top 15 BVs recommended by the lowest densities of nurses with the condition that the actual rates of PAHs are higher than 4.5 PAHs per 1,000 inhabitants. As it can be seen through the maps, the BVs selected by approach using SVR are different to the ones selected by the descriptive statistic methods. In addition, as our algorithm also returns the rates of PAH reduction per to-be-added nurse, we can assess the effectiveness of the approach using SVR 11
Fig. 5. BVs to increase nurses and the best number of nurses to add for the biggest reduction of PAH recommended by SVR
Fig. 6. BVs to increase nurses recommended by the high rates of PAH
Fig. 7. BVs to increase nurses recommended by the low densities of nurses
by comparing it with the two descriptive statistic methods. For example, in the Table 2, if we increase 9 nurses (number of nurses - No Nurses in Table 2), we expect the number of PAHs to be reduced is 6.3 (No PAHs in Table 2), and therefore the rate of PAH reduction per to-be-added nurse is 6.3/9 = 0.7 (Reduction Rate in Table 2) Although we can not compute the rates of PAH reduction per to-be-added nurse for the descriptive statistic methods, but if we use the reduction numbers of PAHs from the approach using SVR, we can have the rates of PAH reduction per to-be-added nurse for the selected BVs as shown in Tables 3 and 4. 12
Table 2. PAH reduction per to-be-added nurse by SVR No Nurses
No PAHs
Reduction Rate
9 15 20 24 30
6.3 9.7 12.4 14.4 17.0
0.70 0.65 0.62 0.60 0.57
Table 4. PAH reduction per to-be-added Table 3. PAH reduction per to-be-added nurse recommended by low densities of nurse recommended by high rates of PAHs nurses No Nurses
No PAHs
Reduction Rate
No Nurses
No PAHs
Reduction Rate
9 14 19 24 30
2.47 5.30 6.09 7.80 9.54
0.27 0.38 0.32 0.32 0.32
10 16 19 25 30
0.42 2.84 3.19 5.29 5.73
0.04 0.18 0.17 0.21 0.19
By comparing the results in the Table 2 with the results in the other Tables (3 and 4), we can somehow confirm the effectiveness of the approach using SVR for selecting the BVs to increase nurses.
5
Conclusions
In this paper, we have presented an application of machine learning to health care services. In particular, after evaluating several regression methods including multilinear regression, k-nearest neighbors, and neural networks, we have chosen support vector machine for regression (SVR) as the best method for the extraction of the list of the cross-border living areas (fr. Bassins de vie - BVs) to recommend to the local health authorities for health care service improvement in general and nurse incremental in particular. The method is able to point out how many to-be-added nurses in each living area (BV) for the biggest reduction of the number of potentially avoidable hospitalizations (PAHs). In addition, in our approach, we take into account the constraints related to the budget (or the maximum number of nurses to be added) and the equality of health care access for the inhabitants in the region regardless of their geographical and socioeconomic situation. In our work, our approach is applied to the Occitanie region, but it can be applied to other regions or extended at the national level or even to other countries. Moreover, this approach could be applied to other health care policy issues, such as the reduction of hospital readmissions or access to innovation. As a result, our approach has led to a start-up project in France. 13
In future works we plan to test new determinants of PAHs related to the environment and weather conditions, such as pollution and temperature. These variables have not been tested before, even though they are linked to the chronic conditions subject to PAHs. Every time we have new data, besides applying SVR, we will also test neural networks for the comparativeness for the best result.
References 1. Segal, M., Rollins, E., Hodges, K., Roozeboom, M.: Medicare-Medicaid eligible beneficiaries and potentially avoidable hospitalizations. Medicare Medicaid Res Rev. 4(1) (2014) 2. Freund, T., Campbell, S., Geissler, S., Kunz, C., Mahler C., Peters-Klimm, F., Szecsenyi, J.: Strategies for reducing potentially avoidable hospitalizations for ambulatory caresensitive conditions. Ann Fam Med. 11(4), 363–370 (2013) 3. Mercier, G., Georgescu V., Bousquet J.: Geographic variation in potentially avoidable hospitalizations in France. Health affairs. 34, 836–843 (2015) 4. T. Ngo, V. Georgescu, T. Libourel, A. Laurent, G. Mercier. Spatial gradual patterns: Application to the measurement of potentially avoidable hospitalizations. In Proc. of the SOFSEM Int. Conf., Austria. 596–608 (2018) 5. Gao, J., Moran, E., Li, Y., Almenoff, P.: Predicting potentially avoidable hospitalizations. Med Care. 52(2):164–71 (2014) 6. Bindman AB, Grumbach K, Osmond D, Komaromy M, Vranizan K, Lurie N, Billings J, Stewart A.: Preventable hospitalizations and access to health care. JAMA.274(4):305–11. (1995) 7. Bourret R., Mercier G., Mercier J., Jonquet O., De La Coussaye JE., Bousquet PJ., Robine JM., Bousquet J.: Comparison of two methods to report potentially avoidable hospitalizations in France in 2012: a cross-sectional study. BMC Health Serv Res.15:4 (2015) 8. Vapnik, V., A. Lerner: Pattern recognition using generalized portrait method. Automation and Remote Control. 24, 774–780 (1963) 9. Trafalis, T.B., Ince H.: Support vector machine for regression and applications to financial forecasting. Proceedings of the IEEE-INNS-ENNS International Joint Conference on Neural Networks. IJCNN 2000. 6, 348–353 (2000) 10. Cortes, C., Vapnik V.: Support vector networks. M Learning. 20(3), 273–297(1995) 11. Smola A.J., Schlkopf, B.: A Tutorial on support vector regression. Statistics and Computing. 14(3), 199–222 (2004) 12. Support Vector Machine - Regression (SVR). http://www.saedsayad.com/ support_vector_machine_reg.htm 13. Chai, T. and Draxler, R. R.: Root mean square error (RMSE) or mean absolute error (MAE)? Arguments against avoiding RMSE in the literature, Geosci. Model Dev., 7, 1247–1250 (2014)
14
