IT Final Essay Assignment Help
|Student ID Number:||Student Surname:||Given name:|
|Program:||Stage / Year:||Module name:|
|Unit code:||Unit title:|
|Due date:||Date submitted:|
|Assignment Title and number:|
|Components of submission|
|Number and type of pieces submitted:|
|Number of pages in the report:|
I declare that this assessment item is my own work unless otherwise acknowledged and is in accordance with the University’s Student Academic Integrity Policy.
strong>Signature of student: _____________________________ Submission date: ___/___/___
The goal of this chapter is to introduce the decription of the methodology and the results from using Systematic Literature Reviw (SLR) and to assess and document the state of the art in e-Portfolio framework. This will point to answer of the research questions that were proposed at the start of the research and will lay the foundation for further research in this domain. The structure and contents of this SLR chapter are taken from the guidelines presented in (Keele, 2007). In section 2 the chapter centers on useful background information that is important for understanding the review. In section 3 that chapter revolves around the research methodology that is used for this review. Section 4 outlines the results and an analysis of the review. Section 5 presents review’s conclusion.
Kitchenham and Charters’ views (Keelem2007) rest on the assumption that show a justification of the need of the review and a summary of other reviews should be undertaken.It is clearly shown in Chapter 2 that there is a lacking categorisation in the literature relating to e-portfolio frameworks and it is also demonstrated the laying a foundation it would be helpful to categorise and systematically analyse the literature in relation to collaborative e-portfolio framework in a scientific way. Therefore it is proposed that a SLR would be a suitable method to do this and would also help establishing gaps in the literature.
A systematic literature review (often referred to as a systematic review) is a mean of analysing, evaluating, identifying and interpreting all available evidence that is related to a particular research questiin, or topic area, or phenomenon of interest. (SLRs) are methods for making sense of large volumes information.They are used to interpret the information in order to explain what does not work when exploring specific search themes, social policy or practical issues (Petticrew and Roberts,2006).
Individual studies that take part to a systematic review are called primary studies while the systematic review is a form of secondary study that uses a well-defined methodology . SLRs originated in medicine where they were used to bridge the gap between knowledge and practice. The method has since filtered into many science and social science disciplines and it is often used to help inform policy-making. In many cases when conducting a review, an SLR uses citation indices, a research protocol, search strings, inclusion and exclusion criteria .
There are many reasons why the researcher adopted the conducting SLR .Firstly, to summarise all existing studies about a collaborative e-Portfolio framework for education ,industry and government. Secondly, to identify any gaps between them. Thirdly, to clarifying the strengths and weaknesses of the studies about the research area .And forthly, to improve our outcomes to investigating the aim of our research which is to design and pilot deployment of a collaborative e-Portfolio framework for use in Saudi Arabia governmental planning to promote lifelong learning in a knowledge economy.
1.3 Research Methodology
This study has been undertaken as a systematic literature review based on the orginal guidelines as proposed by Kitchenham ]22[. In this case the goal of the review is to asses systematic literature revies (which are referred to as secondary studies). This section will describe the design of the view , the execution of the SLR and the threats to the validity of this review. The research process used has been taken from the guidelines set out by Kitchenham and Charters (Keele, 2007) and the researcher (as a single researcher) has undertaken the ‘light’ version of the review guidelines. The steps in the systematic literature review method are documented below :
|1. The Need for the SLR|
2. Research Questions
3. Develop the Review Protocol
4. Validate the Review protocol
There are three main phases of the reviw and 10 steps assoociated with each phase that are highlighted in Figure 1.. Section 3.1 describes planning the review.Planning the review is about developing and validation the review protocol .Section 3.2 describes conducting the review. Conducting that is about starting with study selection and results in extracted data and synthesized information Section 3.3 describes reporting the review.Reporting the view that is about writing and validating the report.
Planning the Review
Figure 1 Phases and Steps of SLR
1.3.1 Planning the Review
The following 4 steps were followed in the planning phase of the review.
184.108.40.206 Step 1: The Need for a SLR
SLR methodology is widely implemented in some disciplines, such as medicine and sociology .Since Kitchenham et al. Published the seminal paper ]15[ of Evidence-Based Engineering (EBSE) in 2004 , systematic review has become an important research methodology of EBSE, and many SLRs have been conducted and reported in Software Engineering (SE) .One of the main goals of an Slr is to ensure that the rview is methodical and repeatable and to minimize the level of bias that can be prevalented in Traditional (ad hoc) Literature Review (TLRS).
There are many reasons for undertaking a systematic literature review. The most common reasons are :
- To summarize the existing evidence concerning a treatment or technology e.g.to summarize the empirical evidence of the benefits and limitations of a specific agile method .
- To identify any gaps in current research in order to suggest areas for further investigation.
- To provide a framework/ background in order to appropriately position new research activities.
However, systematic literature reviews can also be undertaken to examine the extent to which the empirical evidence /contradicts theoretical hypotheses, or even to assist the generation of new hypotheses.
In the following research, the need for the review was demonstrated by searching The Evidence Based Software Engineering (EBSE) website (2012) that include a database for a record of any SLR on the devlopment of the software process that is used in a game dvelopment .One these grounds , we can argue the consensus view seems that the researcher was not found existing SLR on the research topic. The reasonable grounds of the need to perform this reviw is to show that the following reserach has not been undertaken perviously. The researcher found no existing SLR on the research topic.
220.127.116.11 Step 2: Research Questions
In formulating the scope for the reviews , the PICOS (participants, intervention, comparators, outcomes and study design structure) may be helpful in framing the review questions. This can help the reviewers to delineate clearly if they wish, for example, to compare and summarise systematic views that address the sam treatment comparison or a particular intervention for a population or condition, or a range of interventions for people with specific condition.
There are four primary review questions and their aims that are answered with specific information and extracted from the primary studies are provided in Table 1. The research questions addressed by this study are:
|RQ1||What are the components of a collaborative e-Portfolio framework that enhance the state of art?||To identify the components, services of a collaborative e-Portfolio framework, to determine the state of art in practice and theory about e-Portfolio.|
|RQ2||What are the stakeholders’ needs and perspectives about e-Portfolio?||To identify all potential stakeholders and users, their current and future needs and perspectives about e-Portfolios and if there is any relationship between them.|
|RQ3||How could a collaborative e-Portfolio framework be used to promote lifelong learning and knowledge economy?||To understand how e-Portfolio are and can be used to promote lifelong learning and knowledge economy, its potential role and stakeholder perspectives|
|RQ4||How can collaborative e-Portfolio framework be used in governmental strategic planning for promoting lifelong learning and knowledge economy?||To understand how collaborative e-Portfolio could be used as governmental strategic planning tool to determine if there are any governmental sectors using e-Portfolios for any reason, To identify the current and future role for governmental planning to promote knowledge economy and lifelong learning ,To understand how governmental strategic planning use collaborative e-Portfolio framework for promoting lifelong learning and knowledge economy.|
The table 1 research questions and their aims for the SLR
18.104.22.168 Step 3 and 4: Develop and Validate a Review Protocol
The protocol is a critical element of any systematic review and it is the paln that describes a conduct of a proposed systematic literature review. A review protocol specifies the methods that will be used to undertake a specific systematic review. A pre-defined protocol is necessary to reduce the possibility of researcher bias .For example, without a protocol, it is possiple that the selection of individual studies or the analysis may be driven by researcher expectations. In medicine, review protocols are usually submitted to pee review.
Researchers must agree a procedure for evaluating the protocol .If appropriate funding is available , a group of independent experts should be asked to review the protocol. The same experts can later be asked to review the final report. PHD students should present their protocol to their supervisors for review and criticism. The basic SLR review questions discussed in the table 1 can be adapted to assist the evaluation of a systematic review protocol. In addition, the internal consistency of the protocol can be checked to confirm that:
- The search strings are appropriately derived from the search questions.
- The data to be extraxted will properly address the research question(s).
- The data analysis procedure is appropriate to answer the research questions.
We need to specify the research questions and the review scope in order to formulate search strings for literature extraction.Refining the protocol will enable to do a pilot study of the systematic review with 50% of the included studies. It is a given that the defintion and the evaluation of the review protocol based on the objectives. The researcher is concerned with the issue of developing a review protocol (Step 3) to decrease researcher bias and to emphasize that the review could be replicated (the process followed for the review protocol is detailed in figure 3 in appendix A).The evaluation of the protocol (Step 4).This study draws on the reserach that is conducted by the researcher’s supervisor and the subsequent process of implementing the review iteratively improved the design of the review. The final review protocol is described in section 3.2.
1.3.2 Conducting the Review
The methods in conducting a systematic review of reviews require consideration of the following aspects , akin to planning for a systematic review of individual studies: sources, review selection, quality assessment of reviews, presenting of results and implications for practice and research .The objective and reasons for conducting a systematic review of reviews should be made explicit at the start of the process , as this likely to influence the methods used for the review.
The following 5 steps were followed in the conducting phase of the review.
22.214.171.124 Step 5: Search Strategy
The aim of search strategy is to ensure that the systematic review of reviews is comprehensive, through and objective.Kitchenham et al.]21[ used their structured questions to construct serach strings for use with electronic databases.The search strings were used on 6 digital libraries That are INSPEC , El Compendex, Science Direct, Web of Science, IEExplore and ACM Digital Library.
Kitchenham et al .used the procedure recommended by most guideline for performing systematic review. However , it resulted in extremely long search strings that needed to be adapted to specify search engines. Jorgensen] 17[ used a databse previously constructed for a wide survey of software cost estimation. This is an example of how valuable a mapping study can be . He also used a fairly simple search string on the INSPEC database. Kitchenham et al attempt to produce a search string that was very specific to their research question but they still found a large number of false positives. In practice, a simpler search string might have been hust as effective. Its important to note that neither study based its search process solely on searching digital libraries. Both studies had very specific research questions and the researchers were aware that the number of papers addressing the topic would be small. Thus, both studies tries hard to undertake a comprehensive search.
The time period for the first and second questions are limited of 2010to 2015 (inclusive). This was agreed in consultation with the researcher’s supervisor. It was also decided that reference lists of primary studies would be checked to find other primary studies; this is referred to as snowballing (EBSE, 2013). Creation of the search protocol consisted of a trial search similar to that performed by Unterkalmsteiner et al. (2012). The process that was followed to identify the relevant research is trial search is illustrated in Figure 2.
Figure 2 trial search
This study draws on research conducted the research questions that helped to specify the initial keywors for the search and to identify candidate primary studies of a serach of the digital libraries with the keywords e-portfolio, eelctronic portfolio, e-folio and digital portfolio in the metdat. We depend on the digital libraries to locate peer reviewed paper. A trail serach was conducted on the science Digital Library and IEE Explore. The search string which is composed of the population AND intervention was iteratively changed until the relevant quota of publications was found. The keywords were iteratively improved until there was a>=90% match rate. The final search strategy is outlined in Table 2.
|Population||e-portfolio or electronic portfolio or e-folio or digital portfolio in the metdat|
|e-portfolio or electronic portfolio or e-folio or digital portfolio in the metdat|
|e-portfolio or electronic portfolio or e-folio or digital portfolio|
in the metdat
|e-portfolio or electronic portfolio or e-folio or digital portfolio in the metdat|
|Intervention||Definition OR components OR service OR system||user OR|
|“Knowledge economy” OR|
“knowledge based economy “OR “learning economy” OR economy OR “work integrated learning” OR education OR “lifelong learning” OR learning OR “knowledge work “OR “life wide learning” OR “lifelong education” OR “continuing professional development”
|government OR planning OR strategic OR policy|
Table 2 Search Strategy
The resulting searches led to 2618 publications and a procedure was put in place to help store, track and reference the studies in an organised and reproducible fashion. The following tools were used by the researcher: Microsoft excel was used to store the references; endnot was used as a reference manager; and Atlas.ti which is a computer program that is used mostly , but not exclusively in qualitative research or qualitative data analysis tool which involved storing the full text studies and data extraction .
The purpose of ATLAS.ti is to help researchers uncover and systematically analyze complex phenomena hidden in unstructed data (text, multimedia, geospatial). The program provides tools that let the user locate, code and annotate findings in primary data material, to weigh and evaluate their importance, and to visualize the often complex relationship between them.ATLAS.ti is used by researchers and practitioners in a wide variety of fields including arthropology, arts, architecture, communication, criminology, economics, educational sciences, engineering, management studies, market research, quality management and sociology. ATLAS.ti consolidates large volumes of documents and keeps track of all notes annotationss, codes and memos in all fields that require close study and analysis of primary material consisting of text, images, audio, video and deo data. The source software (Aquad Computer Aided Textual Markup& Analysis Coding Analysis Toolkit Compendium RQDA. Wikipedia).
126.96.36.199 Step 6: Primary Study Selection
A major challenge to review selection is identifying all reviews relevant to the topic of interest, and of potential importance to answering the research question. During the planning phase, before commencing the systematic review of reviews, a review team should be established. The review team should include at least one person with methodological expertise in conducting systematic reviews and at least one person with expertise on the topic under review. The review team is responsible for developing a review selection strategy. An agreement of inclusion and exclusion criteria should be made before starting the review selection process. Aspects of this process might include decisions regarding the type of reviews that may be included in the systematic review. For example, in our review on interventions for preventing preterm birth , we restricted the inclusion criteria to reviews of randomized controlled trials. Another example of inclusion criteria might be to limit the systematic review of reviews to reviews of a particular type of participant (such as women having their first baby) or which assess a particular type of pain relief.
When a selection strategy has been developed, the selection process is carried out in a similar way to a review of individual studies:
- Assess retrieved titles and abstracts for relevance and duplication.
- Select those you wish to retrieve and appraise further.
- Obtain full text copies of these potentially eligible reviews.
- Assess these reviews for relevance and quality; ideally, using independent assessment by at least two members of the review team. This reduces bias in review selection and allows for appropriate discussion should uncertainty arise.
Selection of the primary studies have a defined study selection criteria and a study selection procedure. For the study selection criteria, the main criterion for the inclusion of a primary study was that it was a peer reviewed journal, conference paper or published book contain qualitative and quantitative research. The inclusion and exclusion criteria are shown in Table 3.
The selection of study procedure:
The study selection is a multistage process. Initially, selection criteria should be interpreted liberally, so taht unless a study iddentified by the electronic and hand searches can be clearly excluded based on title and abstract , a full copy should be obtained. However, Bereton et al ]5[ point out that “The standard of IT and software engineering abstracts is too poor to rely on when selecting primary studies. You should also review the conclusions.” The next step is to apply inclusion/ exxlusion criteria based on practical issues ]11[ such as : Language, Journal, Authors, Setting , Participants or subjects , Research Design, Sampling method and Data of publication.
Staples and Niazi point out this sometimes necessay to consider the questions that are not being addressed in order to refine the exclusion criteria ]27[ . Example Staples and Niazi’s research question way 19
- Why do organizations embark on CMM-based SPI initiative? They also defined comlementary research questions that were not being investigatd.
- What motivates individuals to support the addition of CMM-Based SPI in an organization?
- Why should organizations emabrk on CMM-based SPI initiative?
- What reasons for embarking on CMM-based SPI are the most important to organizations?
- What benefits have organizations received from CMM-based SPI initiatives ?
- How do organizations decide to embark on CMM-based SPI intiatives ?
- What problems do organizations have at the time that they decide to adopt CMM-based SPI?
This clarifies the boundaries of their research question of interest .For example, they were concerned with the motivations of organizations not the motivations of individuals and they were concerned with why organizations rejected CMM not whey they adopted it .They found that this process directly improved and clarified their primary study selection and data extraction process.
For the sake of previous discussion, I would like to recommend that maintaing a list of exclused papers, only after the totally irrelevant papers have been excluded, in particular, maintaining a record of those candidate primary studies taht are excluded as a result of the more detailed inclusion / exclusion criteria
This pilot helped to ensure that the study selection criteria and the study classification were consistent between the researcher and the supervisor. A study selection pilot and a data extraction pilot (step 5) as was performed by Unterkalmsteiner et al. (2012). The pilot study involved the first 20 results returned from a search of the Science Direct on-line digital library. The selection criterion was applied on the title, abstract and if necessary on the introduction and conclusion to ascertain if the primary study addressed the research questions. This helped refine the research questions and the inclusion and exclusion criteria. A flowchart of the process followed for the pilot of the systematic review is shown in figure 4 in appendix B.
There was satisfactory agreement, as illustrated by a Cohen Kappa (Emam, 1999) value of 0.6 Cohen’s kappa coefficient is a statistical measure of interHYPERLINK “http://en.wikipedia.org/wiki/Inter-rater_agreement”-HYPERLINK “http://en.wikipedia.org/wiki/Inter-rater_agreement”rater agreement for qualitative (categorical) items, and is thought to be more robust than percentage agreement as it takes agreement by chance into account. The equation for the coefficient is: .
|No.||inclusion criteria||exclusion criteria|
|1||The main objective of the paper to discuss or investigate issues related to collaborative e-Portfolio framework.||Papers discusses e-Portfolio principle, create, design, open source or not related to the research questions|
|2||Must have e-Portfolio context||Papers for which only PowerPoint presentation or extended abstracts were available which means abstracts which don’t provide sufficient information|
|3||relates to one or more of the research questions||Papers presenting results without providing evidence|
|4||Must be written in English or Arabic|
Table 3 Lists of inclusion and exclusin criteria
The hypothetical probability of chance agreement Pr (e) =0.6 the relative observed agreement among the rates Pr (a) =0.9, and ? = 0.6 (two places of decimal). Conflicts in the results were resolved with a post mortem and this helped fine-tune the inclusion and exclusion criteria. The selection procedure started at this point. Searching the digital libraries to find relevant papers and more papers were found by following up references. It was found by the researcher that a good way of finding further references was using a facility in the databases for identifying related research. A total of 2618papers were retrieved from the searches of the digital libraries. Duplicates, Non-English texts and unavailable studies were excluded and each set was reduced accordingly to the full text studies of 162 papers, as illustrated in Table 4.
|Round 1 (Title,|
Keyword and abstract)
|Round 2 (Full text)|
Duplicate and unavailable removed
Table 4 Primary study selection
188.8.131.52 Step 7: Study Quality Assessment
In addition to general inclusion/ exclusion criteria, it is considered critical to assess the “quality” of primary studies.
- To provide still more detailed inclusion/ exclusion criteria
- To investigate whether quality differences provide an explanation for difference in study results.20.
- As a means of weighting the importance of individual studies are being synthesised
- To guide the interpretation of findings and detemine the strength of inferences
- To guide recommendations for further research.
An intial difficulty is that there is no agreed definition of study “qulaity”. However, the CRD Guidlines ]19[ and the Cochrane Reviewers’ Handbook ]7[ both suggest taht the quality relates to the extent to which the study minimise bias and maximises internal and external validity .
When assessing the quality of the reviews, one should try to avoid being influenced by extraneous variables, such as authors, institutional affiliations and journal names; and should focus on the conduct of the review.
The study selection phase can be used in two ways: The first way is the initial selection. The second one is the final selection .
- Initial Selection: This selection helps to assist primary study selection in which the data are used to construct detailed inclusion/exclusion criteria .The objectives of the first quality assessment in this review (step 7 of the protocol) help to further refine the inclusion/exclusion criteria. This was done prior to data extraction and limited primary quantitative studies to those that had clear aims.
- Final Selection: This selection helps to asist data analysis and synthesis in which the data is based on a quick scan of the studies to observe proper validation. In this case the quality data are used to identify subsets of the primary study associated with areas identified for further research. The quality data can be collected at the same time as the main data extraction activity using a joint form. The objectives of the second quality assessment in this review help to identify subsets of the primary studies to investigate if the quality differences are associated with the different primary study outcomes and to help with guiding recommendations for further research.
This SLR contained both qualitative and quantitative studies and according to Kitchenham and Charters (Keele, 2007) there must be a separate quality instrument for each. In this research the quality data was used to help identify suitable subsets of primary studies and could help guide recommendations for further research. The quality data was collected at the same time as the main data extraction activity. A joint form was used for both data extraction and quality data extraction. Based on quality instrument used by Unterkalmsteiner et al. (2012) the following questions seemed appropriate for each quantitative study (Table 5) and based on the quality instrument used by Beecham (2008) for qualitative studies the following questions seemed appropriate (Table 8).
|Question||Quantitative Studies (145 Studies)||Answer|
|1.||Are the study aims clearly stated?||Yes/Partially/No|
|2.||Is the method for collecting sample data specified?||Yes/Partially/No|
|3.||Is there any statistical assessment of result?||Yes/Partially/No|
|4.||Are threats to validity considered?||Yes/Partially/No|
Table 5 Quality Assessment Questions for Quantitative Primary Studies
|Question||Qualitative Studies (17 Studies)||Answer|
|1.||Are the study aims clearly stated?||Yes/Partially/No|
|2.||Is enough evidence provided to support the claims?||Yes/Partially/No|
|3.||Is the paper well referenced?||Yes/Partially/No|
Table 6 Quality Assessment Questions for Qualitative Primary Studies
The questions were scored as follows: QA1: Y (yes), the inclusion criteria are explicitly defined in the study, P (Partly), the inclusion criteria are implicit; N (no), the inclusion criteria are not defined and cannot be readily inferred.
Once a primary study passed the inclusion and exclusion assessments its quality was assessed. Qualitative and quantitative (See table 9 and 10 in appendix E) studies had key questions answered during the data extraction. There was a pilot of the two quality checklists (to assess for reliability and usability before applying to all the primary studies) and this was performed at the same time as the data extraction pilot and followed the same protocol. The quality checklists were done primarily to provide more detailed inclusion and exclusion criteria, Primary studies were only accepted if the aims of the paper were clearly stated in the case of quantitative studies. Quality checklists can also be used to partition the primary studies during aggregation, and to explain the results of primary studies. What questions were in each were based on bias and validity of studies. The questions were chosen to reflect what would be significant at the data synthesis stage. Overall it was envisioned by the researcher that the study quality assessment would be of significance in guiding future research. The researcher adopted a suggestion from (Fink, 2013) to review the list of questions available in various reviews in the context of this specific study and select those quality evaluation questions that were most appropriate for the researcher’s specific review questions. The researcher constructed a measurement scale for each item since sometimes a simple Yes/No answer could have been misleading.
184.108.40.206 Step 8: Data Extraction
The objective of this stage is to design data extraction forms to accuratley record the information researchers obtain from the primary studies.To reduce the opportunity for bias . Data extractions forms should be defined and puloted when the study protocol is designed to collect all the information needed to address the review questions and the study quality criteria .In most cases, data extration will define a set of numerical values that should be extracted for ecah study (e.g.number of subjects, treatment effect, confidence intervals,etc.) Nu,erical data are important for any attempt to summarise the results of a set of primary studies and a prerequisite for meta-analysis(i.e.statistical techniques aimed at intergrating the results of the primary studies).Data extraxtion forms need to be piloted on a sample of primary studies.
If several researchesr will use the forms, they should all take part in th pilot. The pilot studies are intended to assess both technical issues such as the completeness of the forms and usability issues such as the clarity of user instructions and the ordering of questions.Electronic forms are useful and can facilitate subsequent analysis .The data extraction instument included a series of structured questions and comprises five sections:i.bibliographic information; ii.focus of the study; iii.methodology; iv.findings; and v.analysis. The series of structured questions within each section were designed to ensure that the review team extracted the data consistently. The quality of the studies and the evidence produced were assessed by an analysis of the strengths and limitations of the empirical studies, which were included in the Methodology section of the data-extraction instrument.The weight of evidence within each study was addressed in the final section of the data-extraction instrument, the analysis section. The aim was to identify the available evidence in support of the review question and sub-question focusing on:
- The curricular and extra curricular interventions identified and their aims ;
- The evidence demonstarting that the curricular or extra-curricular interventions assist career decision making, learning and occupational progression; for example:Wallace,A., Croucher,k., Quilgars, D.and Baldwin, S.(2004).
One researcher extracted teh data and another checked teh extraction. The procedure of having one extractor and one checker is not consistent with the medical standards summarized in kitchenham’s guidelines]22[ .Kitchenham coordinated the data extraction and checking tasks. A pilot of the data extraction on 10 primary studies was carried by the researcher to check for usability, correctness and completeness. The data extraction form was iteratively improved until a full understanding and consensus was reached.
that evolved to the final data extraction model that catered for what emerged during data extraction phase. This was done based on the method used by Staples and Niazi (2007). The full and final data extraction form is shown in table 3 in Appendix D and contains all the properties that were included in the data extraction; this includes the study methodological quality that was collected at the same time as the main data extraction. The data extraction properties relevant to this review are illustrated in Table 11. The studies were classified similar to Unterkalmsteiner (2012) according to their context: Industry that refers to studies in where the research was performed in collaboration with or embedded in industry; or Non-industry that refers to studies performed in an academic setting or where the research environment is not properly described.
|Research Year||Overview of the studies|
|Research Method||Overview of the studies|
|What are the components of a collaborative e-Portfolio framework?||RQ1|
|What are the stakeholders’ needs and perspectives about ePortfolio?||RQ2|
|How could a collaborative e-Portfolio framework be used to promote lifelong learning?||RQ3 (a)|
|How could a collaborative e-Portfolio framework be used to promote knowledge economy?||RQ3(b)|
|How collaborative e-Portfolio could be used as governmental|
strategic planning tool to determine if there are any governmental sectors using e-Portfolios for any reason?
|Identify the current and future role for governmental planning to promote knowledge economy and lifelong learning?||RQ4(b)|
|How governmental strategic planning use collaborative ePortfolio framework for promoting lifelong learning and knowledge economy?||RQ4(c)|
Table 7 Data Extraction Form
The primary studies were those that were selected in the initial pilot study (step 5). A second data extraction pilot was performed to check for consistency of the data extraction and that the studies were classified correctly. This was repeated until there was inter rate agreement between the researcher and supervisor using Cohen Kappa (Emam, 1999). As a single researcher it was important to make sure that the process of extracting data from the included primary studies was validated. An independent validation of data extraction is recommended by Kitchenham and Charters (Keele, 2007). This validation was done to ensure a homogenous interpretation of the data that needed to be extracted from each primary study and a validation and consensus of any categorisation that might be present. This was done based on a random sample of 2 papers and did show up discrepancies in the interpretation of the data properties labelling. The labelling was corrected and the data extraction was performed on the full set of collected studies.
220.127.116.11 Step 9: Data Synthesis
Data analysis was tabulated to show : The number of SLRs publishe per year and their resource.
In ordr to answer the research questions , the properities should be extracted and tabulated . The quantitive and qualitative studies were synthesized separetely .
Synthesis according to the EBSE (2012) is ‘The process of systematically combining different sources of data (evidence) to answer a research question’. Synthesising primary studies is mandatory for a systematic review otherwise it is no more than a mapping study (Cruzes and Dyba, 2011b). The methods of synthesis used in this review are presented to show how knowledge was built and conclusions were reached. The researcher synthesized the quantitative and qualitative studies separately (Keele, 2007). The analysis of the findings was presented in tabular form. Data synthesis brought together all the findings reported in the selected primary studies that had data extracted (see tables in appendix F)
It was proposed by the researcher that the quality data collected could be used for Sensitivity analysis. Sensitivity analysis is important whether you have undetaken a descriptive or quantitative synthesis. However, it is usually easier to perform as a part of a meta analysis (since quantitative sensitivity analysis techniques are well understood). In such case, the results of the analysis should be repeated on various subsets of primary studies to determine whether the result are robust.
Sensitivity analysis procedure aimed at ssessing whether the result of a systematic literature review or meta-analysis are unduly influenced by a small number of studies.Sensitivity analysis methods involve assessing the impact of high leverage studies (e.g.Large studies or studies with atypical results), and ensuring that overall results of a systematic literature remain the same if low quality studies (or high quality) studies are omitted from the analysis, or analysed separately (klee,2007).
In this research sensitivity analyses (highlighting similarities and differences between groups) was performed for all review questions based on: Study approach (e.g. qualitative versus quantitative studies), Context (Industrial versus Non-Industrial).
The studies were demonstrated that the industry refers to studies in where the research was performed in collaboration with or embedded in industry, or non-industry that refers to studies performed in an academic setting or where the research environment isnot properly described.
1.3.3 Reporting the Review
The following steps was followed in the Reporting phase of the review.
18.104.22.168 Step 10: Validation and Study Report
The final phase of the review included the study report (see Section 3.4) and validation. Primary threats to validity identified by the researcher were: There are some fine lines on what is included and what is not. Boundaries of certain topics are changing
Each digital library is unique and has its own search engine which means that there is no specific search strategy, each search criteria has to be tailored to the specific library. This can be quite subjective in terms of the overall principles of a SLR.
Due to the broad scope and complexity of the review it was not possible to manually collate and analyse all the data. Therefore Atlas TI was employed as a qualitative analysis tool.
The number of studies in 2008 is a threat as the researcher was undertaking the study in this year. To help minimise the chance of any new study appearing the researcher has set up search alerts in all of the digital libraries. There was no mechanism for including any new primary study in the SLR during the write up stage of the review.
Extracting the required properties usually involves ‘data extraction forms’ to help structure this process, and ideally uses two analysts working independently to ensure consistency. The researcher used a test retest approach to help counteract this threat.
A total of 162 primary studies were collated and analysed as shown in figure 3 .The researchers were publishe in 2008 . There were many research methods recorded; case studies being the most frequently used. The majority (85%) of the primary studies emanated fro research/academia and are referred to as non-industrial (N). The balance, (15%), was industrial (I).
Figure 3 Analysis of Primary Studies
1.4.1 Findings .
22.214.171.124 (RQ1) what are the components of a collaborative e-Portfolio framework?
A total of 89 studies were analysed to identify the components, services of a collaborative e-Portfolio framework and to determine the state of art in practice and theory about e-Portfolio. The primary studies were categorized according to quantitative studies (35%) (As shown in table 13 appendix F) in which the components of electronic porflio as following:-
1-Resume/reference in the academic 100% , in the industry 100% and the total 100%.
2-The comonents of the projects in the academics 100%, in the industry 100% and in th total 100%.
3-written work in the academics 100%, in the industry 97% and the total 98%.
Appendix C is shown P which are the references of primary studies. The most studies were non-industrial (98%) and the rest of studies were industrial by (2%).P….. Is example for an industrial context. P….. Is example for quantitative studies, a non-industrial context and P? Is example for qualitative studies, a non-industrial context.
126.96.36.199 (RQ2) what are the stakeholders’ needs and perspectives about e-Portfolio?
A total of 59 studies were analysed to identify all potential stakeholders and users, their current and future needs and perspectives about e-Portfolios and if there is any relationship between them. The primary studies were categorized cording to quantitative studies (24%) (As shown in table… appendix F) and qualitative studies (86%) (As shown in table appendix F) and were analysed according to context. Appendix C is shown P 24 which are the references of primary studies. The studies were categorised according context a non-industrial (95%) and an industrial by (5%).P24 .The example for an industrial context is the example for quantitative studies, and a non-industrial context Is example for qualitative studies.
188.8.131.52 (RQ3) how could a collaborative e-Portfolio framework be used to promote lifelong learning in technology and economy?
A total of 27 studies were analysed to understand how e-Portfolio are and can be used to promote lifelong learning and knowledge economy, its potential role and stakeholder perspectives. The most primary studies were referred to how could a collaborative e-Portfolio framework that is used to promote Lifelong learning and how it could promote knowledge economy.
e-portfolios can support life-long learning by supporting the electronic transfer of learning records.It is inevitable to have self-reformation, discover the facilities of technology and use them in both private and work life. People record their detailed curriculum vitae, personal development and new acquisitions under the name of “portfolio”, “personal output folder” or “personal development folder”. These folders are recorded in digital setting and even made available by means online access; which is called “e-portfolio”. Thinking “lifelong learning” concept in parallel with “e-portfolio”.
184.108.40.206 (RQ4) how can collaborative e-Portfolio framework be used in governmental strategic planning for promoting lifelong learning and knowledge economy?
We addressed a research question 4 to understand how collaborative e-Portfolio could be used as governmental strategic planning tool ; to determine if there are any governmental sectors using e-Portfolios for any reason; to identify the current and future role for governmental planning and to promote knowledge economy and lifelong learning and to understand how governmental strategic planning use collaborative ePortfolio framework for promoting lifelong learning and knowledge economy.
How can collaborative EPF be used in government stragic planning for promoting LLL and KE?
|Stud y ID||Context|
|Year of study||Study Approach|
(Qualitativ e/Quantitati ve)
|Notes (if any)|
1.4.2 Analysis of the reviews
220.127.116.11 Strengths and Weaknesses
The discussion has provided a sound evidence base on considerations in the systematic review is a mean of analysing, evaluating, identifying and interpreting all available evidence that is related to a particular research questiin, or topic area, or phenomenon of interest. (SLRs) are methods for making sense of large volumes information.The dicussion is in Section 3.3.1. In addition to this an appraisal of the primary studies for inclusion in the SLR is deemed important by the researcher because the quality of the study is affected by the results of the individual primary studies, as are the conclusions derived from the data synthesis (Cruzes and Dyba, 2011b). A transparent quality evaluation helps the researcher to set a minimum quality baseline for the inclusion of an individual study, helps discriminate between overall contributions of studies and helps the researcher gauge the strength of evidence.
The underpinning principles and key concepts used wer proposed by the researcher that the quality data collected could be used for Sensitivity analysis. This is an analysis procedure aimed at assessing whether the results of a systematic literature review are unduly influenced by a small number of studies. This did not transpire in this research as it was not required at this stage of the research. There are reviews that do not undertake a specific quality assessment of the primary studies for example (Jorgensen, 2009).
Overall the findings of the study quality data are a useful guide for the researcher of what is of significance in the study quality data for future research h in terms of the fact that the studies with lower quality results should be of less significance in the future research.
Appraising the quality of SLR’s is done by a toolkit provided by (Centre for Reviews and Dissemination, 2013). Kitchenham et al. used the DARE criteria outlined next to evaluate the quality of SLR’s in SE (2009):
- Were inclusion/exclusion criteria reported?
- Was the research adequate?
- Were the included studies synthesised?
- Was the validity of the included studies assessed?
- Are sufficient details about the included studies presented?
The researcher proposes that the SLR adequately investigates and reports on all 5 criteria for the current research.
18.104.22.168 Meaning of Finding
In this section I will summarize the ground covered in the
1.4.1 Findings section. The SLR was adopted to ansewer the specific question about the components of a collaborative Portfolio framework, the stakeholders’ needs and perspectives about ePortfolio, could a collaborative e-Portfolio framework be used to promote lifelong learning and knowledge economy.
(RQ2) what are the stakeholders’ needs and perspectives about e-Portfolio? Stakeholders have a variety of needs. So, it is very important that you consider that a crucial element of any e-portfolio implantation into your organization is to identify internal and external project stakeholder and to address their needs. Understanding how these stakeholder can both contribute to the e-portfolio implementation process and also benefit from e-portfolios is very important: It provides information about The design of e-portfolio and how these stakeholder can be integrated into the culture of the institution .
More information about the different perspectives of stakeholder and how they perceive their
potential benefits you can find on the web page of the Australian e-portdolio project , carried out
Queensland University of Technology .
More information about the different perspectives of stakeholder and how they perceive their potential benefits you can find on the web page of the Australian e-Portfolio project, carried out by Queensland University of Technology: Further information on strategies for identifying and understanding stakeholders you can find in a video in the presentation files by Helen L. Chen and Tracy Penny-Light, available at ]3[.
(RQ4) How can collaborative e-Portfolio framework be used in governmental strategic planning for promoting lifelong learning and knowledge economy and technology? The findings of the review are a foundation to answer the above questions and are a means of addressing some of the overall aims of the study, such as addressing the lack of research that exists in the role for governmental planning to promote knowledge economy and lifelong learning. The results are generalizable within the context of this review and are set to act as a foundation for future research. There is the potential for many avenues of future research based on the findings and the gaps identified in this review.
The concept of an e-portfolio has been most advanced in formal education and training. However, it has implications well beyond how its is used in the assessment of learning.
“Electronic potfolios have a greater potential to alter higher education at its very core than any other technology application we have known thus far.” (Trent Baston, 2003, The Electronic Portfolio: What is it All About?). Baston (2003) defines an e-portfolio system as “a dynamic wb site taht interfaces with a database of student work artifacts”. It ls, in other words , like a repository (see also the chapter in this book on institutional repositories) except that its is specifically focussed on products created by students.
e-Portfolios can support life-long learning by supporting the electronic transfer of learning records. This means that prior learning and achievements are taken into account and then after graduating students can take their records with them into employment. In today’s world, with the expansion of technology, new pursuits and in parallel with that , new concepts such as life long learning and e-portfolio emerge. Ever-growing technology removes the borders of learning and results in continuous learning need. That’s why learning is defined as “from cradle to grave” .It is inevitable to have self-reformation, discover the facilities of technology and use them in both private and work life. People record their detailed curriculum vitae, personal development and new acquisitions under the name of “portfolio”, “personal output folder” or “personal development folder”. These folders are recording in digital setting and eben made available by means of online access; which is called “e-portfolio”. Thinking “lifelong learning” concept in parallel with “e-portfolio”
On the other hand, the process of creating electronic portfolios continue after graduation, meanwhile there must be a mediated monitor, which are in this case government recruitments departs.
Institutions such as the American Association of Colleges and Universities have focused on e-Portfolios in courses, programs, learning outcomes, and student evaluation. E-portfolio can be defined as a collection of student work over a period of time. An electronic portfolio uses electronic technologies, allowing the portfolio developer to collect and organize portfolio artifacts in many media types (audio, video, graphics, text). It can be said that electronic portfolios are considered to be a helpful tool for students in which they can include reflective summaries of their achievements, skills, and works. They also help them in their future jobs; especially when searching for one.Electronic portfolio, mostly containing Pictures and Hyperlinks as types of media, in the format of Pdfs, PowerPoint Presentations and Word Documents, should basically include information about the students, such as: personal information, technology skills, content of teaching and field study abilities.
The research examined the perspectives of the Academics and the Industry towards electronic portfolio, and the result was that both of them believe that it is a Sonnet and a Map (Map: Creating a plan and setting goals, Sonnet: Provides a framework, but the content can showcase creativity and diversity). However, there is a strong positive relationship between the Academics and Industry perspectives of electronic portfolios and needs, in that the higher the rate is with the Academics, the higher it also gets with Industry. This helps them both collaborate together, in the sense that the Academics increases employability through providing Industry with electronic portfolios that facilitate the process of seeking employees.
There was a lack of research to answer RQ4 to understand how collaborative e-Portfolio could be used as governmental strategic planning tool and to understand the role for governmental planning to promote knowledge economy and lifelong learning .This shows the need for research in this area.
22.214.171.124 The implications of the results of this SLR
This SLR looked at the published studies relating to the components, services, stakeholders and users e-portfolios used in a collaborative e-Portfolio framework and is significant in that it provided supporting evidence for the proposition about the lack of research in the literature on components, services, stakeholders and users e-portfolios used in an e-portfolio framework combining educational and training institutions, industry and government. The SLR helped establish gaps in the literature relating to how e-Portfolio are and can be used to promote lifelong learning and knowledge economy and how governmental strategic planning use collaborative e-Portfolio framework for promoting lifelong learning and knowledge economy.The focus of this research is on the collaborative e-portfolio framework between different sectors and as such it would be beneficial to explore if there are any governmental sectors using e-Portfolios for any reason.
126.96.36.199.3 Questions Unanswered and Implications for Future Research .
There was a lack of studies answering how collaborative e-Portfolio could be used as governmental strategic planning tool and future role for governmental planning to promote knowledge economy and lifelong learning. Overall there is a need to research To identify the current and future role for governmental planning to promote knowledge economy and lifelong learning and understand how governmental strategic planning use collaborative e-Portfolio framework for promoting lifelong learning and knowledge economy.
There is a description of many components of e-portfolio in the the following (RQ1,RQ2,RQ3,RQ4). (RQ1) the stakeholders (RQ2) used in a collaborative e-portfolio frame work and there were not enough information about how e-Portfolio are and can be used to promote lifelong learning and knowledge economy (RQ3) there were no information about the role for governmental planning to promote knowledge economy and lifelong learning (RQ4) There is a gap in this area in the research.
The central points in understanding the previous descriptions are how collaborative e-Portfolio could be used as governmental strategic planning tool and identifying the current and future role for government planning to promote knowledge economy ; Depending on lifelong learning to understand how governmental strategic planning use collaborative e-Portfolio framework for promoting lifelong learning and knowledge economy are important to design an e-portfolio framework combining educational and training institutions, industry and government.
This chapter addressed the review questions .This identified gaps in the research literature and laid the foundation (in terms of the need) for further research in investigating to design a collaborative e-portfolio framework for education and training institutions, industry and governmental institutions like the Ministry of Civil Service in Saudi Arabia according to the requirements and components of a collaborative e-portfolio framework.
Sensitivity analyses (highlighting similarities and differences between groups) will be performed for all research questions based on: Study Type (e.g. quantitative versus qualitative studies), Context (industrial versus non-industry) and Development Approach (Agile versus Hybrid).