The use of information and information gain in the analysis of attribute dependencies

• Autorzy: Krzysztof Moliński , Anita Dobek , Kamila Tomaszyk
• Języki publikacji: EN

Abstrakty

EN

This paper demonstrates the possible conclusions which can be drawn from an analysis of entropy and information. Because of its universality, entropy can be widely used in different subjects, especially in biomedicine. Based on simulated data the similarities and differences between the grouping of attributes and testing of their independencies are shown. It follows that a complete exploration of data sets requires both of these elements. A new concept introduced in this paper is that of normed information gain, allowing the use of any logarithm in the definition of entropy.

Słowa kluczowe

EN

dendrogram; entropy; information gain

• Wydawca: De Gruyter Open
• Czasopismo: Biometrical Letters
• Rocznik: 2012
• Tom: 49
• Numer: 2
• Strony: 149-158

Daty

wydano

2012-12-01

online

2013-08-17

Twórcy

autor

Krzysztof Moliński

• Department of Mathematical and Statistical Methods, Poznań University of Life Sciences, Poznań, Poland

autor

Anita Dobek

• Department of Mathematical and Statistical Methods, Poznań University of Life Sciences, Poznań, Poland

autor

Kamila Tomaszyk

• Department of Mathematical and Statistical Methods, Poznań University of Life Sciences, Poznań, Poland

Bibliografia

• Bezzi M. (2007): Quantifying the information transmitted in a single stimulus. Biosystems, 89: 4-9.[Crossref][WoS]
• Brunsell N.A. (2010): A multiscale information theory approach to assess spatial- temporal variability of daily precipitation. Journal of Hydrology 385: 165-172.[WoS]
• Jakulin A. (2005). Machine learning based on attribute informations. PhD Dissertation. University of Ljubljana.
• Jakulin A., Bratko I., Smrke D., Demsar J., Zupan B. (2003): Attribute interactions in medical data analysis. In: 9th Conference on Artificial Intelligence in Medicine in Europe (AIME 2003), October 18-22, (2003), Protaras, Cyprus.
• Jakulin A., Bratko I. (2003): Analyzing attribute dependencies. In: 7th European Conference on Principles and Practice of Knowledge Discovery in Databases (PKDD 2003), September 22-26, Cavtat, Croatia.
• Jakulin A., Bratko I. (2004a): Quantifying and visualizing attribute interactions: an approach based on entropy. http://arxiv.org/abs/cs.AI/0308002v3.
• Jakulin A., Bratko I. (2004b): Testing the significance of attribute interaction. Proc. 21st International Conference on Machine Learning. Banff, Canada.
• Kang G., Yue W., Zhang J., Cui Y., Zuo Y., Zhang D. (2008): An entropy-based approach for testing genetic epistasis underlying complex diseases. Journal of Theoretical Biology 250: 362-374.[WoS]
• Kullback S., Leibler R.A. (1951): On information and sufficiency. Annals of Mathematical Statistics 22(1): 79-86.[Crossref]
• Matsuda H. (2000): Physical nature of higher-order mutual information. Intrinsic correlation and frustration. Physical Review E, 62: 3096-3102.
• McGill W.J. (1954): Multivariate information transmission. Psychometrika 19(2): 97-116.[Crossref]
• Moniz L.J., Cooch E.G., Ellner S.P., Nichols J.D., Nichols J.M. (2007): Application of information theory methods to food web reconstruction. Ecological Modeling 208: 145-158.
• Moore J.H., Gilbert J.C., Tsai C.-T., Chiang F.-T., Holden T., Barney N., White B.C. (2006): A flexible computational framework for detecting, characterizing, and interpreting statistical patterns of epistasis in genetic studies of human disease susceptibility. Journal of Theoretical Biology 241: 252-261.
• Rajski C. (1961): A metric space of discrete probability distributions. Information and Control 4: 373-377.
• Shannon C. (1948): A mathematical theory of communication. Bell System Technical Journal 27: 379-423, 623-656.
• Yan Z. Wang Z., Xie H. (2008): The application of mutual information-based feature selection and fuzzy LS-SVM-based classifier in motion classification. Computer Methods and Programs in Biomedicine 90: 275-284.[WoS]

Identyfikatory

DOI

10.2478/bile-2013-0011