# Bayesian approach to the process of identification of the determinants of innovativeness.

IntroductionOur previous research was devoted to application of self-organizing feature maps in correlating groups of time series to the process of country-level innovativeness assessment and also to the area of entrepreneurship which we think is the crucial factor that enables the development of innovative economies (Czyzewska et al. 2012a; Czyzewska et al. 2012b). Before our experiments there was no research, in the available literature, applying anything other than traditional statistical methods of analysis in the field such as univariate and multivariate statistics, e.g. regression analysis which is mostly used to quantify an individual variable's impact on the overall innovativeness performance (Coad et al., 2014; Meuer et al.,2014). Additionally, the methods of linear ordering of objects were used, i. e. by Grzelak and Starzynska (2014).

In this paper a new approach was applied involving the usage of Bayesian networks in the process of the economy innovativeness assessment. There is a complex set of factors describing the innovativeness of economies.

Our goal in this paper is to identify the most important factors establishing the innovativeness level of national economies and to measure the strength of each determinant influencing the overall innovativeness performance of a country. We selected the elements affecting the innovativeness level of European Union national economies by determining the strength of the indicators' influence in the Summary Innovation Index (SII). We based our analysis on the Innovation Union Scoreboard (IUS) data from 2006 to 2014.

In our investigations we used a special computer tool called BeliefSEEKER (a belief network system) that was developed at the University of Information Technology and Management in Rzeszow, Poland, in cooperation with the University of Kansas. We would like to present the method's application to economic sciences to visualize its potential in multidimensional and complex analysis of the innovativeness of economies.

DATASET APPLIED

In our analysis we used the dataset known as the Innovation Union Scoreboard (Figure 1). The IUS includes innovation indicators and trend analysis for the EU27 Member States, as well as for Croatia, Iceland, the Former Yugoslav Republic of Macedonia, Norway, Serbia, Switzerland and Turkey. It also includes comparisons based on a reduced set of indicators between the EU27 and their 10 global competitors. The IUS replaced the European Innovation Scoreboard which was published from 2001 to 2009.According to the IUS, the EU countries are divided into four groups:

1) innovation leaders (their performance is 20% or more above the average of the EU27),

2) innovation followers (it is less than 20% above but more than 10% below the average of the EU27),

3) moderate innovators (it is less than 10% but more than 50% below the average of the EU27),

4) modest innovators (it is below 50% that of the EU27).

The countries' performance is described by 3 groups of indicators--enablers, firm activities, and outputs, organized in 8 dimensions (Figure 2):

1) human resources,

2) open, excellent and attractive research systems,

3) finance and support,

4) firm investments,

5) linkages and entrepreneurship,

6) intellectual assets,

7) innovators,

8) economic effects.

The Enablers cover the drivers of innovation performance external to the firm and capture three innovation dimensions: 'Human resources', 'Open, excellent and attractive research systems' as well as 'Finance and support'.

Firm activities describe the firm's innovativeness efforts and are grouped in three innovation dimensions: 'Firm investments', 'Linkages & entrepreneurship' and 'Intellectual assets'.

Outputs present the effects of the firm's innovation activities in two innovation dimensions: 'Innovators' and 'Economic effects'.

The innovation dimensions' sub-indices characterizing the national level economic innovativeness performance in more details are outlined below.

Enablers are represented by the following set of indicators:

1) Human resources--the indicator is composed of three sub-indicators:

a. new doctorate graduates (ISCED 6). The indicator measures the supply of new second-stage tertiary graduates.

b. population with completed tertiary education. It shows the number of persons aged 30-34 having completed tertiary education. As the share of population is narrow, the indicator reflects changes in educational policies leading to the increase of tertiary graduates.

c. youth with upper secondary level education--the indicator measures the qualification level of the population aged 20-24 years in terms of formal educational degrees. It is considered to be a very important condition for building a knowledge-based society.

2) Open, excellent and attractive research systems--captures the following sub-indices:

a. international scientific co-publications. The indicator measures the quality of scientific research as the collaboration in research as increases in scientific productivity.

b. scientific publications among top 10% most cited. This is a proxy for the efficiency of the research system as highly cited publications are assumed to be of a higher quality.

c. non-EU doctorate students--the indicator reflects the mobility of students. Attracting doctorate students will secure a net brain gain and inflows of researchers.

3) Finance and support--covering the following indices:

a. public R&D expenditure--this measure is assumed to be the major driver of economic growth that improves production and stimulates growth.

b. venture capital--it reflects the dynamism of new businesses creation. For risky businesses (usually innovative) venture capital is one of the most important resources of business expansion.

The full set of indicators called "Firm activities" is as follows:

1) Firm investments described by:

a. business R&D expenditure as % of GDP (the indicator captures the formal creation of new knowledge within firms. It is particularly important in the science-based sectors: pharmaceuticals, chemicals and some areas of electronics, where new knowledge is created in R&D laboratories or in close cooperation with them).

b. non-R&D innovation expenditure as % of total turnover (the indicator includes investment in equipment and machinery and the acquisition of patents and licenses, as well as measures the diffusion of innovations).

2) Linkages and entrepreneurship developed by:

a. SMEs innovating in-house as % of SMEs (the indicator measures the degree to which SMEs introduce new or significantly improved products or production processes that have innovated in-house).

b. innovative SMEs co-operating with others (% of all SMEs). The indicator shows the degree to which SMEs are involved in innovation co-operation. It shows the flow of knowledge between public research institutions and private companies, and also between companies.

c. public-private co-publications. It presents public-private research linkages and collaboration between business sector and public sector researches resulting in academic publications.

3) Intellectual assets:

a. PCT patent applications per billion GDP (in PPP[euro])--informs about the number of Patent Cooperation Treaty (PCT) patent applications.

b. PCT patent applications in societal challenges per billion GDP (in PPP[euro])--the indicator measures PCT applications in health technology and climate change mitigation.

c. community trademarks per billion GDP (in PPP[euro])--the indicator represents trademarks valid across the European Union registered with the Office for Harmonization in the Internal Market.

d. community designs per billion GDP (in PPP[euro])

--designs valid across the European Union registered at the Office for Harmonization in the Internal Market.

Outputs are represented by:

1) Innovators:

a. SMEs introducing product or process innovations as % of SMEs (the indicator reflects the introduction of new products or services and processes in manufacturing among SMEs).

b. SMEs introducing marketing/organizational innovations as % of SMEs (the indicator captures non-technological innovation among SMEs --introduced in marketing and within their organizations).

2) Economic effects:

a. employment in knowledge-intensive activities as % of total employment (knowledge-intensive activities are defined as those industries where at least 33% of employment have a university degree --ISCED5 or ISCED6).

b. medium and high-tech product exports as % of total products exports (measure of technological competitiveness of the EU, i. e., the ability to commercialize the results of R&D and innovation in international markets).

c. knowledge-intensive services exports as % of total services exports (measure of the competitiveness of the knowledge-intensive services sector).

d. sales of new-to-market and new-to-firm innovations as % of turnover (the indicator shows the share of new or significantly improved products in total turnover).

e. license and patent revenues from abroad as % of GDP (the indicator captures disembodied technology and also other types of innovations acquisition from abroad).

In the paper we applied Bayesian networks to identify and organize the indicators described above according to the importance of their influence on the overall innovativeness performance of an individual country. We would like to introduce the method's application in economic sciences and present it as a new approach to analyzing innovativeness appraisal, which is a multidimensional and complex phenomenon. Further, we provide a brief introduction to the interpretation and semantics of belief networks.

BAYESIAN NETWORKS

Jensen (Jensen, 2001) has defined a Bayesian network -also known as Bayesian Belief Network or Bayes Net--as a set of nodes (i.e. variables, attributes) A1, A2, Am and a set of directed arcs between them. Each variable (node) contains a finite set of mutually exclusive states (values) a1, a2, ..., am. The nodes and arcs form a directed, acyclic graph (Figure 1). Hence, for each structure built from a variable

A and its parents Parent(A1), ... , Parent(Ai) there is an associated potential table P(A\Parent(A1), ... , Parent(Ai)), containing probabilities of all node values for each combination of its parents' values. This means that one can specify the conditional probability distribution (CPD) of the node for given values of its parents.

The notations ai and ~ai are used to indicate Ai = true and Ai = false, respectively.

The set of directed connections (arcs) in the network defines a hierarchy of nodes. If there exists an arc going out from node Ai to node Aj, then we say that Ai is a parent of Aj, or Aj is a child of Ai. The arcs are used to model the probabilistic influences between the variables. The intuitive meaning of an arc in the network corresponds to the statement that Ai has a direct influence on Aj. Absence of an arc between Ai and Aj means that the corresponding variables do not influence each other directly. More formally, a variable Ai is taken to be dependent of its parents and children in the digraph, but is conditionally independent of any of its non-descendants given its parents; this property is commonly known as the Markov condition (Glymour, 1999; Cowell et al., 1999).

Additionally, the network structure describes the causal relationships between network attributes (nodes), so that joint probability distribution P(A1 = a1, ... ,Am = am) is not a product of independent probabilities but is expressed by the following relationship:

P(A1 = [a.sub.1], ..., [A.sub.m] = [a.sub.m]) = [[PI].sup.n.sub.i=1] P(Ai = [a.sub.i]|Parents (ai)

where Parents(Ai) are the nodes preceding node Ai, connected to Ai by causal arcs in the graphical model. It means that for each variable Ai, there is a specified set of probability distributions P(A^Parents(Ai)) describing the joint effect of a specific combination of values for the parents Parents(Ai) of Ai on the probability distribution of Ai.

Thus, the global semantics of the causal network provides information about the joint probability distribution as a product of local conditional distributions, which can be used to calculate the value of probability for any node. Additionally, for each element of this relationship there is a conditional probability table (CPT) associated. Individual rows in this table contain information about the conditional dependence of a node on its predecessors (parents). In the case of nodes with no predecessors, the rows in the table contain information about the a priori probability. The total sum of the values in each row of the table must be equal to 1, because the individual items represent a comprehensive set of values of a given variable. Each element of the joint probability distribution is the product of the corresponding values taken from CPT tables of the Bayesian network. Therefore it can be said that CPT tables are a decomposed representation of the joint probability. Furthermore, Bayesian networks are based on the assumption of independence of nodes, so the network structure is essential for specifying the intransitive dependencies and provides information about the formation of probability distribution.

Bayesian networks can be constructed manually or learned from data. Manual construction of a network involves the following development stages: selection of relevant variables (Shwe et al., 1991; Korver & Lucas, 1993), identification of the relationships among the variables (Gaag & Helsper, 2002), identification of qualitative probabilistic and logical constraints (Renooij & Gaag, 2002), assessment of probabilities (Gaag et al., 2002) and sensitivity analysis and evaluation (Coupe' & Gaag, 2000). For each of these stages, knowledge is acquired from experts in the domain of application, the relevant literature is studied, and available data are analyzed (Ramoni & Sebastiani, 1999).

With the increasing availability of data, learning evidently is a more feasible alternative for developing a Bayesian network. The Bayesian network learning problem can be categorized as 1) a parameter learning problem when the structure is known, and 2) a structure learning problem when the structure is unknown. Generally, the parameter learning is a part of the latter and it is used as an inner loop of the structure learning in the score-and-search-based approach. The mentioned approach comprises of:

1) greedy search with an ordering on the variables (Cooper & Herskovits, 1992; Bouckaert, 1993, 1994; de Santana et al., 2007a; Liu et al., 2007; Liu & Zhu, 2007a, 2007b),

2) greedy search with no ordering on the variables (Lam & Bacchus, 1993, 1994a; Suzuki, 1999; Chickering et al., 1997a, 1997b; Steck, 2000; Hwang et al., 2002),

3) genetic and evolutionary algorithms (Larranaga et al., 1996a, 1996b; Faulkner, 2007),

4) particle swarm optimization (Kennedy & Eberhart, 1995, 1997; Xing-Chen et al., 2007; Li et al., 2006; Sahin & Devasia, 2007),

5) simulated annealing (Kirkpatrick et al., 1983; de Campos &Huete, 2000),

6) other heuristics (Peng& Ding, 2003; de Campos et al., 2002a; Burge & Lane, 2006).

The greedy search provides a way toobtain a good model in a reasonable time frame as compared to other methods. For a fixed amount of computational time, a greedy search with random restarts produces better models than either simulated annealing or best-first search does (Chickering, 2002). In our research, Bayesian belief networks are developed with the help of a heuristic algorithm using the Bayesian function of network structure to distribution matching as a scoring function, named K2 (Jensen, 2001).

To sum up, it can be said that a Bayesian network consists of two basic components: one which is qualitative, which is the graphical structure of model dependencies, that which is quantitative, represented by the probability distributions related to the graph. The feature distinguishing Bayesian networks from other methods of knowledge representation is the number of inference methods. By focusing on the qualitative description (i.e. on the graphical structure of the model) we can identify conditional dependencies between variables. Given the quantitative descriptions (parametric models assigned to nodes), after introducing a new evidence to the model we can obtain a posteriori probability distributions of individual variables of the model, or the joint distribution of a variable set. Based on an expert's opinion we may update probabilities of variable states or values. We can also find the most probable configuration (for the available evidence) of unobserved variables, as well as estimate a hypothesis probability with regard to specific observations. It can therefore be concluded that a Bayesian network seems to be a very useful tool to select the most important factors determining the innovativeness level of national economies and also to arrange the determinants in order of their importance.

APPLIED SUPERVISED MACHINE LEARNING TOOL

BeliefSEEKER is a system developed for supervised machine learning. The software allows us to generate belief networks, applying various algorithms (Grzymala-Busse et al.,2005). Learning models in the form of Bayesian belief networks are developed with the help of a heuristic algorithm using the Bayesian function of network structure to distribution matching as a scoring function, named K2 (Jensen, 2001). The network generation process is performed by searching for a structure which maximizes the scoring function--marginal likelihood--defined as follows:

[MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII]

where:

i = 1, ... , v, where v is the number of nodes in the network,

j = 1, ... , q, the number of possible combinations of parents of node X (if a given attribute does not contain nodes of the "parent" type, then qi assumes the value of 1),

k = 1, ... , c, where ci is the number of classes within attribute [X.sub.i],

[n.sub.ijk] is the number of rows in the data set, for which parents of attribute [X.sub.i], have value j, and this attribute has the value k, and [[alpha].sub.ijk] and [[alpha].sub.ij] are parameters of the initial Dirichlet's distribution (Heckerman, 1999).

It should be stressed that the calculation of Dirichlet's parameter ([alpha]) has been favorably optimized by cutting down the number of iteration steps, owing to the application of a special algorithm for elimination of variables (Jensen, 2001). Currently, BeliefSEEKER allows for development of a single (optimal) belief network (for any given, single value of [alpha]), or it can generate a set of belief networks for an incrementally increased value of a. In a separate process of global optimization, only dissimilar networks are kept for further processing, i.e. generation of belief rules and/ or classification of unseen cases. Belief networks can be developed using the K2 algorithm. The result of the algorithm is a Bayesian network. The learning pro-cess can be done in two ways: (i) maximization of conditional probabilities of training data, known as the ma-ximum likelihood rule, or (ii) the maximum a posteriori probability rule. The first approach entails choosing a hypothesis for which observing training data is the most probable. The second approach is based on Bayes' theorem and requires determining a posteriori probabilities of all hypotheses and choosing the one for which the probability is the highest. These two approaches are available in the BeliefSEEKER system. However, the first one is treated as an alternative. Moreover, the learning method which chooses the most probable hypothesis is also used in classifying new cases. A characteristic feature of the system is the implementation of the original algorithm for converting a Bayesian network to a set of IF...THEN type rules. The elaborated methodology aims to extend the possibilities of phenomenal interpretation of a learning model--generated in the form of a traditional belief network by turning it into a set of rules, hereinafter referred to as belief rules (Mroczek et al., 2004; Grzymala-Busse et al.,2007).

GENERAL METHODOLOGY OF THE RESEARCH

In our research we used the dataset of Innovation Union Scoreboard 2014, containing indicators measuring innovativeness and trend analyses for the EU27 Member States as well as for Croatia, Iceland, the Former Yugoslav Republic of Macedonia, Norway, Serbia, Switzerland and Turkey. The original data set was incomplete: about 200 attribute values were missing. First, the missing attribute values were replaced. For any case x and attribute a with a missing attribute value, we restricted our attention to all cases from the class of x, and the missing attribute value for the attribute a was replaced by an average value of a restricted to the given class.

To select the most important factors determining the innovativeness level of national economies, the research was performed in two main phases. In the first phase, the number of innovation dimensions was reduced. For this purpose, a set of composite indicators was used. By applying the BeliefSEEKER system, a set of Bayesian networks was generated, considering each year separately (i.e. the first network was generated on the basis of composite indicators from the year 2006, the second from the year 2007, etc.) and choosing different values for Dirichlet's parameter *.

The main difference between the generated networks (i.e. learning models) was the type of indicator shaving direct/indirect influence on the Summary Innovation Index. A qualitative analysis of the generated belief networks enabled us to define a set of the most important dimensions. From among 8 dimensions of innovations, 5 were selected. As the experiments have not proved any strong influence of three of the dimensions on national innovativeness level, we excluded from further analysis: innovators, human resources and economic effects (see Table 1).

In the second phase of our research the innovation dimensions' sub-indices of the selected 5 dimensions were analyzed in order to choose the most important factors determining innovativeness performance and to validate their overall importance. It turned out that for the innovation dimensions' sub-indices of the selected dimensions there are no data for years 2011 to 2013. E.g. for the Research systems indicator consisting of 3 sub-indices only the following data were available: International scientific co-publications (from 2005 to 2012), Scientific publications among top 10% most cited (from 2002 to 2010), Non-EU doctorate students (from 2006 to 2011). In this phase we have focused on data from 2006 to 2010 because they were available for all sub-indices of the selected indicators.

Using the previously described approach, for each year and different value of Dirichlet's parameter a Bayesian network was generated. Next, the output learning models were tested using unknown data (i. e. if a learning model was built engaging data from the year 2010, the quality of the model was tested by using data from the years respectively: 2009, 2008, 2007, 2006). From among the group of belief networks, for each year, the network characterized by the lowest classification error (i.e. the highest classification efficiency) was selected (see Table 2).

At the end of the research the five resulting belief networks were analyzed in order to identify the most important descriptive attributes, and to validate their importance in the overall innovativeness performance. Results of this step are shown in Table 3.

DISCUSSION AND CONCLUSIONS

According to Table 3 we can state that the most important factors determining the overall innovativeness performance of national economies are: business expenditures on R&D, PCT patent applications and then cooperation of SMEs in introducing innovations (the importance of this indicator became visible in the two last years of the research period only).

Research systems have indirect influence on the innovativeness level--especially important are the international scientific co-publications and public R&D expenditures, and then scientific publications among top 10% most cited and public-private co-publications.

The quality of the research system--the role of a country's scientists on the worldwide arena--is a significant condition enabling them to introduce highly advanced solutions in the marketplace. Thus international cooperation in research is essential, but what is more important--the scientific cooperation between the public and the private sector is also an element conditioning the quality of solutions introduced in business and also influencing the economies' innovativeness level.

To sum up, we can say that worldwide intellectual assets protection, investing in R&D by companies, and their cooperation in the process of introducing innovations seem to be the crucial factors determining the innovativeness level of an individual country's economy.

These results allow us to formulate recommendations regarding a specific country's innovativeness policy. In further research we plan to explore other datasets, related to the area, to phrase a set of recommendations that can support policymakers' decisions or practitioners engaged in innovative economy creation.

Although in the last few years, the Bayesian probabilistic reasoning method has gained more popularity, it is not often used by many scientists mostly by those involved in data mining and artificial intelligence. For this reason, in further research we will try to explain the networks by their properties' characteristics in the creation of the explanation of quantitative properties in a manner closer to human perceptive abilities (Mroczek & Hippe, 2014).

Received: 02.04.2014

Accepted: 09.07.2014

REFERENCES

Bouckaert, R. R. (1993). Probabilistic Network Construction Using the Minimum Description Length Principle. In Symbolic and Quantitative Approaches to Reasoning and Uncertainty: European Conference, Lecture Notes in Computer Science 747: 41-48.

Bouckaert, R. R. (1994). Probabilistic Network Construction Using the Minimum Description Length Principle. Technical report RUU-CS-94-27, Department of Computer Science, Utrecht University.

Burge, J., Lane, T. (2006). Improving Bayesian Network Structure Search with Random Variable Aggregation Hierarchies. In Proceedings of the Seventeenth European Conference on Machine Learning, Lecture Notes in Artificial Intelligence 4212: 66-77.

Chickering, D. M., Heckerman, D., Meek, C. (1997). A Bayesian Approach to Learning Bayesian Networks with Local Structure. In Proceedings of the Thirteenth Annual Conference on Uncertainty in Artificial Intelligence. Morgan Kaufmann, pp. 80-89.

Chickering, D. M., Heckerman, D., Meek, C. (1997). A Bayesian Approach to Learning Bayesian Networks with Local Structure. Technical report MSR-TR-97-07, Microsoft Research.

Chickering, D. M. (2002). Optimal Structure Identification with Greedy Search. Journal of Machine Learning Research 3: 507-55.

Coad, A., Cowling, M., Nightale, P., Pellegrino, G., Savona, M., Siepel, H. (2014).UK Innovation Survey. Innovative Firms and Growth, Science and Technology Policy Research.

Cooper, G. F., Herskovitz, E. (1992). A Bayesian Method for the Induction of probabilistic networks from data. Mach Learn 9: 309-47.

Coupe' V. M. H., van der Gaag, L. C. (2000). Sensitivity Analysis: An Aid for Probability Elicitation. KnowlEng Rev 15: 215-32.

Cowell, R. G., Dawid, A. P., Lauritzen, S. L., Spiegelhalter, D. J. (1999). Probabilistic Networks and Expert Systems. New York: Springer.

Czyzewska, M., Pancerz, K., Szkola, J. (2012).Towards Assessment of Indicators Influence on Innovativeness of Countries' Economies: Selected Soft Computing Approaches. In World Academy of Science, Engineering and Technology, 70: 115-118.

Czyzewska, M., Pancerz, K., Szkola, J. (2012).Self-Organizing Feature Maps in Correlating Groups of Time Series: Experiments with Indicators Describing Entrepreneurship. In Proceedings of the Workshop on Concurrency, Specification and Programming, ed. L. Popova-Zeugmann, 1: 73-78.

de Campos, L. M., Huete, J. F. (2000). Approximating Causal Orderings for Bayesian Networks using Genetic Algorithms and Simulated Annealing. In Proceedings of the Eight Conference on Information Processing and Management of Uncertainty in Knowledge-Based Systems, Madrid, Spain, pp. 333-340.

de Campos, L. M., Ferna'ndez-Luna, J. M., Ga'mez, J. A., Puerta, J. M. (2002). Ant Colony Optimization for Learning Bayesian Networks. International Journal of Approximate Reasoning 31(3): 291-311.

de Santana, A. L., Frances, C. R., Rocha, C. A., Carvalho, S. V., Vijaykumar, N. L., Rego, L. P., Costa, J. C. (2007a). Strategies for Improving the Modeling and interpretability of Bayesian networks. Data and Knowledge Engineering 63(1): 91-107.

Faulkner, E. (2007). K2GA: Heuristically Guided Evolution of Bayesian Network Structures from Data. In Proceedings of the IEEE Symposium on Computational Intelligence and Data Mining, IEEE, pp. 18-25.

van der Gaag, L. C., Helsper, E. M. (2002). Experiences with Modelling Issues in Building Probabilistic Networks. In Proceedings of EKAW on Knowledge Engineering and Knowledge Management: Ontologies and the Semantic Web, Gomez-Perez, A., Benjamins, V. R. (eds). Lecture Notes in Artificial Intelligence 2473: 21-26.

van der Gaag L. C., Renooij, S., Witteman, C. L. M., Aleman, B. M. P., Taal, B. G. (2002). Probabilities for a Probabilistic Network: A Case Study in Esophageal Cancer. ArtifIntell Med 25: 123-148.

Glymour, C., Cooper, G. F. (1999). Computation, Causation, Discovery. Menlo Park, CA: MIT Press.

Grzelak, M. M., Starzynska, W. (2014). Multivariate Statistical Analysis of Innovativeness of Manufacturing Companies in Poland--Selected Aspects. In XXI Journals ASEPUMA--IX EncuentroInternacionalAnales de ASEPUMA no 21: 306.

Grzymala-Busse, J. W., Hippe, Z. S., Mroczek, T. (2005). System BeliefSEEKER--A New Approach to Induction of Belief Networks and Belief Rules. In Artificial Intelligence Methods (AI-METH), Burczynski, T., Cholewa, W., Moczulski, W., 59-60, Silesian University of Technology Edit. Office.

Grzymala-Busse, J. W., Hippe, Z. S., Mroczek, T. (2007). Deriving Belief Networks and Belief Rules form Data: A Progress Report, In: Transactions on Rough Sets VII: Commemorating the Life and Work of Zdzislaw Pawlak, Part II (Lecture Notes in Computer Science), Peters, J. F., Skowron, A., Marek, V. W, Orlowska, W., Slowinski, W., Ziarko, W., 53-69, Berlin-Heidelberg: Springer-Verlag.

Heckerman, D. (1999). A Tutorial on Learning with Bayesian Networks. In Learning in Graphical Models, ed. M. Jordan, MIT Press, Cambridge, MA.

Hwang, K. B., Lee, J. W., Chung, S. W., Zhang, B. T. (2002). Construction of Large-scale Bayesian Networks by Local to Global Search. In Trends in Artificial Intelligence: Proceedings of the Seventh Pacific Rim International Conference on Artificial Intelligence, Lecture Notes in Artificial Intelligence 2417: 375-384.

Innovation Union Scoreboard 2014, European Union 2014, http: //ec.europa.eu/enterprise/policies/innovation/ policy/innovation-scoreboard/index_en.htm.

Jensen, F. V (2001). Bayesian Networks and Decision Graphs. New York: Springer-Verlag.

Kennedy, J., Eberhart, R. C. (1997). A Discrete Binary Version of the Particle Swarm Optimization Algorithm. In Proceedings of the IEEE International Conference on Systems, Man and Cybernetics 5, IEEE, pp. 4104-4108.

Kirkpatrick, S., Gelatt, C. D. Jr., Vecchi, M. P. (1983). Optimization by Simulated Annealing. Science 220(4598): 671-680.

Korver, M., Lucas, P. J. F. (1993). Converting a Rule-based Expert System into a Belief Network. Medical Informatics 18(3): 219-24.

Lam, W., Bacchus, F. (1993). Using Causal Information and Local Measures to Learn Bayesian Networks. In Proceedings of the Ninth Conference on Uncertainty in Artificial Intelligence, Heckerman, D., Mamdani, A. (eds). Morgan Kaufmann, pp. 243-250.

Lam, W., Bacchus, F. (1994). Learning Bayesian Belief Networks: an Approach Based on the MDL Principle. Computational Intelligence 10(3): 269-293.

Larranaga, P., Kuijpers, C. M. H., Murga RH, Yurramendi Y (1996) Learning Bayesian network structures by searching for the best ordering with genetic algorithms. IEEE Transactions on Systems, Man and Cybernetics Part A 26(4): 487-493.

Larranaga, P., Poza, M., Yurramendi, Y., Murga, R. H., Kuijpers, C. M. H. (1996). Structure Learning of Bayesian Networks by Genetic Algorithms: a Performance Analysis of Control Parameters. Transactions on Pattern Analysis and Machine Intelligence 18(9): 912-926.

Li, X. L., Wang, S. C., He, X. D. (2006). Learning Bayesian Networks Structures Based on Memory Binary Particle Swarm Optimization. In Proceedings of the Sixth International Conference on Simulated Evolution and Learning, Lecture Notes in Computer Science 4247: 568574.

Liu, F., Tian, F., Zhu, Q. (2007). An Improved Greedy Bayesian Network Learning Algorithm on Limited Data. In Proceedings of the Seventeenth International Conference on Artificial Neural Networks (ICANN 2007), Lecture Notes in Computer Science, Part I 4668: 49-57.

Liu, F., Zhu, Q. (2007). The Max-relevance and Min-redundancy Greedy Bayesian Network Learning Algorithm. Lecture Notes in Computer Science, Part I 4527: 346-356.

Liu, F., Zhu, Q. (2007). Max-relevance and Min-redundancy Greedy Bayesian Network Learning on High Dimensional Data. In Proceedings of the Third International Conference on Natural Computation, Lei J, Yoo J, Zhang Q (eds), IEEE, pp. 217-221.

Kennedy J, Eberhart R (1995) Particle swarm optimization. In Proceedings of the IEEE International Conference on Neural Networks 4, IEEE, pp 1942-1948

Meuer, J., Rupietta, C. (2014). Backes-gellner U, Idiosyncrasies within and Interdependencies between Innovation Systems: A Mixed Method Approach. Paper presented at the DRUID Society Conference 2014, CBS, Copenhagen, June 16-18.

Mitchell, T. (1997). Machine Learning. M. McGraw Hill.

Mroczek, T., Grzymala-Busse, J. W, Hippe, Z. S. (2004). Rules from Belief Networks: A Rough Set Approach. In Rough Sets and Current Trends in Computing, Tsumoto, S., Slowinski, R., Komorowski, J., Grzymala-Busse, J., 483-487, Berlin-Heidelberg: Springer-Verlag.

Mroczek, T., Hippe, Z. S. (2014). Conversion of Belief Networks into Belief Rules: A New Approach, Bulletin of the Polish Academy of Sciences (in review).

Peng, H., Ding, C. (2003). Structure Search and Stability Enhancement of Bayesian Networks. In Proceedings of the Third IEEE International Conference on Data Mining, Wu, X., Tuzhilin, A., Shavlik, J. (eds.). IEEE Computer Society, pp. 621-624.

Ramoni, M., Sebastiani, P. (1999). Bayesian Methods for Intelligent Data Analysis. Intelligent Data Analysis: An Introduction, Berthold, M., Hand, D. J. (eds.). New York: Springer-Verlag, pp. 128-166.

Renooij, S., van der Gaag, L. C. (2002). From Qualitative to Quantitative Probabilistic Networks. In Darwiche, A., Friedman, N. (eds.). Proceedings of the 18th International Conference on Uncertainty in Artificial Intelligence. San Francisco, CA: Morgan Kaufmann, pp. 422-9.

Russell, S. J., Norvig, P. (2002). Artificial Intelligence: A Modern Approach (2nd ed.). New Jersey: Prentice Hall.

Sahin, F., Devasia, A. (2007). Distributed Particle Swarm Optimization for Structural Bayesian Network Learning. In Swarm Intelligence: Focus on Ant and Particle Swarm Optimization, Chan, F. T. S., Tiwari, M. K. (eds.). I-Tech Education and Publishing, chapter 27, Vienna, Austria, pp. 505-532.

Shwe, M. A., Middleton, B., Heckerman, D. E., Henrion, M., Horvitz, E. J., Lehmann, H. P., Cooper, G. F. (1991). Probabilistic Diagnosis using a Reformulation of the INTERNIST-1/QMR knowledge base. I. The Probabilistic Model and Inference Algorithms. Methods of Information in Medicine, 30(4): 241-255.

Steck, H. (2000). On the Use of Skeletons when Learning in Bayesian Networks. In Proceedings of the Sixteenth Conference on Uncertainty in Artificial Intelligence, Boutilier, C., Goldszmidt, M. (eds.). Morgan Kaufmann, pp. 558-565.

Suzuki, J. (1999). Learning Bayesian Belief Networks Based on the MDL Principle: an Efficient Algorithm using the Branch and Bound Technique. IEICE Transactions on Information and Systems E82-D(2): 356-367.

Xing-Chen, H., Zheng, Q., Lei, T., Li-Ping, S. (2007). Research on Structure Learning of Dynamic Bayesian Networks by Particle Swarm Optimization. In Proceedings of the IEEE Symposium on Artificial Life, IEEE, pp. 85-91.

* The variable a occurring in the scoring function of the K2 algorithm. The research performed showed that controlled modification of the parameter's value has significant influence on the structure of generated belief networks.

Marta Czyzewska *, Teresa Mroczek **

* Ph.D. Marta Czyzewska, Department of Microeconomics, University of Information Technology and Management in Rzeszow, Sucharskiego Street 2, 35-225 Rzeszow, e-mail: mczyzewska@wsiz.rzeszow.pl.

** Ph.D. Teresa Mroczek, Department of Expert Systems and Artificial Intelligence, University of Information Technology and Management in Rzeszow, Sucharskiego Street 2, 35-225 Rzeszow, e-mail: tmroczek@wsiz.rzeszow.pl.

Table 1: Location of indicators (dimension) in the belief networks (+ means direct influence on SII, ++ and +++ means indirect (second and third generation, respectively) relative to SII, and lack of + means no influence on SII Cpmposite indicators 2006 2007 (dimensions) [alpha]=1 [alpha]=4 [alpha]=1 [alpha]=3 0 0 0 Human_resources ++ +++ ++ Research_systems ++ ++ ++ + Finance_and_support ++ + ++ ++ Firm_investments ++ ++ ++ ++ Linkages_&_ + + + + entrepreneurship Intellectual_assets + + + Innovators + Economic_effects Cpmposite indicators 2008 2009 (dimensions) [alpha]=1 [alpha]=1 [alpha]=6 [alpha]=2 00 0 0 Human_resources ++ + + +++ Research_systems + + + ++ Finance_and_support ++ + +++ Firm_investments ++ + + + ++ Linkages_&_ + + + + entrepreneurship Intellectual_assets + + + + Innovators Economic_effects + Cpmposite indicators 2010 2011 (dimensions) [alpha]=4 [alpha]=1 [alpha]=1 [alpha]=6 0 0 0 Human_resources ++ ++ Research_systems + ++ ++ ++ Finance_and_support +++ ++ ++ ++ Firm_investments ++ ++ ++ Linkages_&_ + + + entrepreneurship Intellectual_assets + + + + Innovators + + Economic_effects Cpmposite indicators 2012 2013 (dimensions) [alpha]=1 [alpha]=2 [alpha]=1 [alpha]=6 0 0 Human_resources ++ ++ + Research_systems + ++ ++ Finance_and_support ++ ++ ++ Firm_investments + + + + Linkages_&_ ++ ++ + entrepreneurship Intellectual_assets + + + + Innovators ++ Economic_effects + Table 2: Results of classification 2006 2007 2008 2009 [alpha]=30 [alpha]=90 [alpha]=20 [alpha]=30 Error Rate 37.86% 31.43% 27.43% 27.57% 2010 [alpha]=40 Error Rate 39.29% Table 3: Location of sub-indices in the belief networks (+ means direct influence on the Summary Innovation Index, ++ and +++ means indirect influence--second and third generation, respectively on the Summary Innovation Index) 2006 2007 [alpha]=30 [alpha]=90 Research International ++ ++ systems scientific co- publications Scientific ++ + publications among top 10% most cited Non-EU doctorate ++ students Finance and Public R&D ++ ++ support expenditure Venture capital + ++ Firm Business R&D + + investments expenditure Non-R&D innovation expenditure Linkages & SMEs innovating entrepreneurs in-house hip Innovative SMEs collaborating with others Public-private co- ++ ++ publications Intellectual PCT patent + + Assets applications PCT patent applications in societal challenges Community + + trademarks Community designs 2008 2009 [alpha]=20 [alpha]=30 Research International ++ + systems scientific co- publications Scientific ++ +++ publications among top 10% most cited Non-EU doctorate ++ students Finance and Public R&D ++ ++ support expenditure Venture capital Firm Business R&D + + investments expenditure Non-R&D ++ innovation expenditure Linkages & SMEs innovating + entrepreneurs in-house hip Innovative SMEs + collaborating with others Public-private co- ++ publications Intellectual PCT patent + Assets applications PCT patent applications in societal challenges Community trademarks Community designs 2010 [alpha]=40 Research International ++ systems scientific co- publications Scientific ++ publications among top 10% most cited Non-EU doctorate ++ students Finance and Public R&D ++ support expenditure Venture capital ++ Firm Business R&D + investments expenditure Non-R&D innovation expenditure Linkages & SMEs innovating entrepreneurs in-house hip Innovative SMEs + collaborating with others Public-private co- + publications Intellectual PCT patent + Assets applications PCT patent applications in societal challenges Community ++ trademarks Community + designs

Printer friendly Cite/link Email Feedback | |

Author: | Czyzewska, Marta; Mroczek, Teresa |
---|---|

Publication: | eFinanse |

Date: | Jun 1, 2014 |

Words: | 5954 |

Previous Article: | Estimation of risk neutral measure for polish stock market. |

Next Article: | Analysis of the privatization process in poland in the years 2008-2011--outcomes and prospects. |

Topics: |