Cross-task code reuse in genetic programming applied to visual learning

• Autorzy: Wojciech Jaśkowski , Krzysztof Krawiec , Bartosz Wieloch
• Języki publikacji: EN

Abstrakty

EN

We propose a method that enables effective code reuse between evolutionary runs that solve a set of related visual learning tasks. We start with introducing a visual learning approach that uses genetic programming individuals to recognize objects. The process of recognition is generative, i.e., requires the learner to restore the shape of the processed object. This method is extended with a code reuse mechanism by introducing a crossbreeding operator that allows importing the genetic material from other evolutionary runs. In the experimental part, we compare the performance of the extended approach to the basic method on a real-world task of handwritten character recognition, and conclude that code reuse leads to better results in terms of fitness and recognition accuracy. Detailed analysis of the crossbred genetic material shows also that code reuse is most profitable when the recognized objects exhibit visual similarity.

Słowa kluczowe

EN

genetic programming; code reuse; knowledge sharing; visual learning; multi-task learning; optical character recognition

• Wydawca: University of Zielona Gora Press
• Czasopismo: International Journal of Applied Mathematics and Computer Science
• Rocznik: 2014
• Tom: 24
• Numer: 1
• Strony: 183-197

Daty

wydano

2014

otrzymano

2013-04-18

poprawiono

2013-09-28

poprawiono

2013-11-07

Twórcy

autor

Wojciech Jaśkowski

• Institute of Computing Science, Poznań University of Technology, ul. Piotrowo 2, 60-965 Poznań, Poland

autor

Krzysztof Krawiec

• Institute of Computing Science, Poznań University of Technology, ul. Piotrowo 2, 60-965 Poznań, Poland

autor

Bartosz Wieloch

• Institute of Computing Science, Poznań University of Technology, ul. Piotrowo 2, 60-965 Poznań, Poland

Bibliografia

• Bhanu, B., Lin, Y. and Krawiec, K. (2005). Evolutionary Synthesis of Pattern Recognition Systems, Springer-Verlag, New York, NY.
• Caruana, R. (1997). Multitask learning, Machine Learning 28(1): 41-75.
• Chang, Y.F., Lee, J.C., Mohd Rijal, O. and Syed Abu Bakar, S.A.R. (2010). Efficient online handwritten Chinese character recognition system using a two-dimensional functional relationship model, International Journal of Applied Mathematics and Computer Science 20(4): 727-738, doi: 10.2478/v10006-010-0055-x.
• Ciresan, D.C., Meier, U. and Schmidhuber, J. (2012). Multi-column deep neural networks for image classification, Proceedings of the 2012 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Providence, RI, USA, pp. 3642-3649.
• Fabijańska, A. (2012). A survey of subpixel edge detection methods for images of heat-emitting metal specimens, International Journal of Applied Mathematics and Computer Science 22(3): 695-710, DOI: 10.2478/v10006-012-0052-3.
• Galvan Lopez, E., Poli, R. and Coello Coello, C.A. (2004). Reusing code in genetic programming, in M. Keijzer, U.-M. O'Reilly, S.M. Lucas, E. Costa and T. Soule (Eds.), Genetic Programming-7th European Conference, EuroGP 2004, Proceedings, Lecture Notes in Computer Science, Vol. 3003, Springer-Verlag, Berlin/Heidelberg, pp. 359-368.
• Ghosn, J. and Bengio, Y. (2003). Bias learning, knowledge sharing, IEEE Transactions on Neural Networks 14(4): 748-765.
• Hall, M., Frank, E., Holmes, G., Pfahringer, B., Reutemann, P. and Witten, I.H. (2009). The Weka data mining software: An update, SIGKDD Explorations 11(1): 10-18.
• Haynes, T. (1997). On-line adaptation of search via knowledge reuse, in J.R. Koza, K. Deb, M. Dorigo, D.B. Fogel, M. Garzon, H. Iba and R.L. Riolo (Eds.), Genetic Programming 1997: Proceedings of the Second Annual Conference, Morgan Kaufmann, San Francisco, CA, pp. 156-161.
• Holland, J. (1975). Adaptation in Natural and Artificial Systems, Vol. 1, University of Michigan Press, Ann Arbor, MI.
• Hornby, G.S. and Pollack, J.B. (2002). Creating high-level components with a generative representation for body-brain evolution, Artificial Life 8(3): 223-246.
• Howard, D. (2003). Modularization by multi-run frequency driven subtree encapsulation, in R.L. Riolo and B. Worzel (Eds.), Genetic Programming Theory and Practice, Kluwer, New York, NY, Chapter 10, pp. 155-172.
• Howard, D., Roberts, S.C. and Ryan, C. (2006). Pragmatic genetic programming strategy for the problem of vehicle detection in airborne reconnaissance, Pattern Recognition Letters 27(11): 1275-1288.
• Hsu, W.H., Harmon, S.J., Rodriguez, E. and Zhong, C. (2004). Empirical comparison of incremental reuse strategies in genetic programming for keep-away soccer, in M. Keijzer (Ed.), Late Breaking Papers at the 2004 Genetic and Evolutionary Computation Conference, Association for Computing Machinery, Seattle, WA.
• Jaśkowski, W., Krawiec, K. and Wieloch, B. (2007a). Genetic programming for cross-task knowledge sharing, in D. Thierens (Ed.), Genetic and Evolutionary Computation Conference GECCO, Association for Computing Machinery, London, pp. 1620-1627.
• Jaśkowski, W., Krawiec, K. and Wieloch, B. (2007b). Knowledge reuse in genetic programming applied to visual learning, in D. Thierens (Ed.), Genetic and Evolutionary Computation Conference GECCO, Association for Computing Machinery, London, pp. 1790-1797.
• Jaśkowski, W., Krawiec, K. and Wieloch, B. (2007c). Learning and recognition of hand-drawn shapes using generative genetic programming, in M. Giacobini (Ed.), EvoWorkshops 2007, Lecture Notes in Computer Science, Vol. 4448, Springer-Verlag, Berlin/Heidelberg, pp. 281-290.
• Koza, J. (1992). Genetic Programming, MIT Press, Cambridge, MA.
• Koza, J.R. (1994). Genetic Programming II: Automatic Discovery of Reusable Programs, MIT Press, Cambridge, MA.
• Koza, J.R., Bennett III, F.H., Andre, D. and Keane, M.A. (1996). Reuse, parameterized reuse, and hierarchical reuse of substructures in evolving electrical circuits using genetic programming, in T. Higuchi (Ed.), Proceedings of International Conference on Evolvable Systems: From Biology to Hardware (ICES-96), Lecture Notes in Computer Science, Vol. 1259, Springer-Verlag, Berlin.
• Krawiec, K. (2006). Learning high-level visual concepts using attributed primitives and genetic programming, in F. Rothlauf (Ed.), EvoWorkshops 2006, Lecture Notes in Computer Science, Vol. 3907, Springer-Verlag, Berlin/Heidelberg, pp. 515-519.
• Krawiec, K. (2007). Generative learning of visual concepts using multiobjective genetic programming, Pattern Recognition Letters 28(16): 2385-2400.
• Krawiec, K. and Bhanu, B. (2005). Visual learning by coevolutionary feature synthesis, IEEE Transactions on System, Man, and Cybernetics, Part B 35(3): 409-425.
• Kurashige, K., Fukuda, T. and Hoshino, H. (2003). Reusing primitive and acquired motion knowledge for gait generation of a six-legged robot using genetic programming, Journal of Intelligent and Robotic Systems 38(1): 121-134.
• Langdon, W.B. and Poli, R. (2002). Foundations of Genetic Programming, Springer-Verlag, New York, NY.
• Li, B., Li, X., Mabu, S. and Hirasawa, K. (2012). Towards automatic discovery and reuse of subroutines in variable size genetic network programming, in X. Li (Ed.), Proceedings of the 2012 IEEE Congress on Evolutionary Computation, Brisbane, Australia, pp. 485-492.
• Louis, S. and McDonnell, J. (2004). Learning with case-injected genetic algorithms, IEEE Transactions on Evolutionary Computation 8(4): 316-328.
• Luke, S. (2002). ECJ evolutionary computation system, http://cs.gmu.edu/eclab/projects/ecj/.
• Mitchell, T.M. (2006). The discipline of machine learning, Technical Report CMU-ML-06-108, Machine Learning Department, Carnegie Mellon University, Pittsburgh, PA.
• Montana, D.J. (1993). Strongly typed genetic programming, BBN Technical Report #7866, Bolt Beranek and Newman, Inc., Cambridge, MA.
• Moya, M.R., Koch, M.W. and Hostetler, L.D. (1993). One-class classifier networks for target recognition applications, World Congress on Neural Networks, Portland, OR, USA, pp. 797-801.
• O'Sullivan, J. and Thrun, S. (1995). A Robot That Improves Its Ability to Learn, Carnegie Mellon University, School of Computer Science, Pittsburgh, PA.
• Poli, R., Langdon, W.B. and McPhee I.N.F. (2008). A field guide to genetic programming, http://www.gp-field-guide.org.uk/.
• Perez, C.B. and Olague, G. (2013). Genetic programming as strategy for learning image descriptor operators, Intelligent Data Analysis 17(4): 561-583.
• Pratt, L.Y., Mostow, J. and Kamm, C.A. (1991). Direct transfer of learned information among neural networks, Proceedings of the 9th National Conference on Artificial Intelligence (AAAI-91), Anaheim, CA, USA, pp. 584-589.
• Rizki, M.M., Zmuda, M.A. and Tamburino, L.A. (2002). Evolving pattern recognition systems, IEEE Transactions on Evolutionary Computation 6(6): 594-609.
• Roberts, S.C., Howard, D. and Koza, J.R. (2001). Evolving modules in genetic programming by subtree encapsulation, in J.F. Miller (Ed.), Genetic Programming, Proceedings of EuroGP'2001, Lecture Notes in Computer Science, Vol. 2038, Springer-Verlag, Berlin, pp. 160-175.
• Rosca, J.P. and Ballard, D.H. (1996). Discovery of subroutines in genetic programming, in P.J. Angeline and K.E. Kinnear, Jr. (Eds.), Advances in Genetic Programming 2, MIT Press, Cambridge, MA, Chapter 9, pp. 177-202.
• Seront, G. (1995). External concepts reuse in genetic programming, in E.V. Siegel and J.R. Koza (Eds.), Working Notes for the AAAI Symposium on Genetic Programming, AAAI/MIT, Cambridge, MA, pp. 94-98.
• Tackett, W.A. (1993). Genetic generation of 'dendritic' trees for image classification, Proceedings of the World Congress on Neural Networks, Portland, OR, USA, pp. IV 646-649.
• Teller, A. and Veloso, M. (1997). PADO: A new learning architecture for object recognition, in K. Ikeuchi and M. Veloso (Eds.), Symbolic Visual Learning, Oxford Press, New York, NY, pp. 77-112.
• Trujillo, L. and Olague, G. (2006). Synthesis of interest point detectors through genetic programming, in M. Cattolico (Ed.), Genetic and Evolutionary Computation Conference GECCO, Association for Computing Machinery, Seattle, WA, pp. 887-894.
• Vilalta, R. and Drissi, Y. (2002). A perspective view and survey of meta-learning, Artificial Intelligence Review 18(2): 77-95.
• Whitley, D., Rana, S. and Heckendorn, R. (1999). The island model genetic algorithm: On separability, population size and convergence, Journal of Computing and Information Technology 7(1): 33-47.

Identyfikatory

DOI

10.2478/amcs-2014-0014