Iterative Feature Selection Taking into Account Their Redundancy: Solving Inverse Problem of Spectroscopy by Neural NetworksстатьяИсследовательская статья
Аннотация:When solving inverse problems of spectroscopy with a neural network, it becomes necessary to reduce the dimension of the input data in order to achieve a more accurate and stable solution while reducing computational complexity. Since the intensities of the spectrum channels are most often used as input features, most of these features may be redundant: on the one hand, some of them may not carry useful information related to the problem being solved, on the other hand, some subsets of features (for example, neighboring channels) may carry similar information. The method used in this study is based on iterative selection of features with the greatest relevance to the target variable and on the exclusion of redundant features with high statistical dependence on each other. In this study, we consider the physical problem of determining the concentration of heavy metal ions in water based on Raman spectroscopy and absorption spectroscopy, as well as their integration, i.e. combining data from both types of spectroscopy. The quality of a neural network solution to a problem based on the full set of input features and on its subsets compiled using the selection method under consideration, as well as using traditional feature selection methods, is compared. In addition, for the proposed feature selection method, various metrics are being considered to determine relevance and redundancy.
