In order to standardize and automatize the time consuming procedures of the meta-analysis, we developed our own metaA (automated meta-analysis) tool. The web-based software includes five basic features: (1) user administration, (2) study editor, (3) study coding, (4) effect size calculator, and (5) meta-analysis calculator. The user administration allows editing and assigning experts or raters to a specific meta-analysis (including username, password, and additional information). With the help of the study editor (see Figure 1), researchers are able to enter all identified studies from the initial literature search result. The studies are specified by publication year, authors, title, and publication type (e.g. journal article, book chapter, dissertation, etc.).

The study coding feature helps raters to code all included studies. The coding of the studies is divided into various sections: theoretical assumptions, sample characteristics (sample size, age, gender, and population type), methodology (research method, setting, design), rating of the studies quality (overall, reliability, validity), subject domain, outcomes (including effect sizes), and additional notes. The effect size calculator enables the raters to calculate the effect size coefficient, Cohen’s d(see Cohen, 1960; Cohen, 1968), for various statistical measures (e.g. t-Test, F-test, correlations, etc.). The meta-analysis calculator finally computes various procedures which enables us to classify the outcomes of the meta-analysis. Results include interrater reliability measures for multiple raters (Fleiss, 1971), mean effect sizes by outcome, and additional descriptive statistics.