Further information, FAQs and resources

Google Scholar, Google Advanced Search, or Google generally, might be appropriate to use depending on the requirements of the systematic review. Google provides a breadth of coverage, however Google does not publishes its reasoning behind what it includes, why it includes it and the ranking of its materials. A systematic review requires a high level of transparency. Therefore, it is important to balance the risk of using materials from Google in a systematic review. 

It is important to recognise the limitations of Google: 

• There is no content coverage list for Google Scholar - there is no “content coverage list” to clearly indicate what is included (and what is not included) in any search made
• Content can be added, but is also removed, without any notice 
• Not all content is indexed to the same standard: Publishers often will allow web crawlers from Google to index the full-text of their content. But not all sources do, which can mean some publications will appear, or be ranked, differently, dependent upon how they are indexed by Google
• You can’t actually view all of your results: It is not uncommon for Google to return searches in there millions. However, no matter how many results you return, Google will never let you see or download more than the first 1,000 results 

Students 

If you are a student undertaking a systematic review and are in need of support, you can book a 1-2-1 with your subject librarian by emailing library@mmu.ac.uk  

Staff 

Subject librarians can meet with programme and unit leaders to discuss the needs of their students and design a bespoke InfoSkills session covering the desired aspects systematic literature reviews, for example, incorporating elements such as MeSH searching, documenting searches etc.  

Researchers 

We are also able to meet with staff on a 1-2-1 basis to discuss their own systematic review needs. Please email your subject librarian or library@mmu.ac.uk 

This largely depends on the aims and scope of your review, and what you are hoping to achieve. 

Library Search is a separate database in its own right and may return results not found elsewhere in other databases, or it may rank the results differently than other databases and search engines. You may therefore discover and find material that you may not have identified elsewhere. However, you may also find cross over with other subject specific databases.  

Library Search can also be useful in finding ‘grey literature,’ such as theses, magazine articles, newspaper articles, reports etc. The content type filters will enable you to filer your results by type of publication. Library Search can be helpful if you have a wide scope and intend to include many different types of publications in your review. 

It can also be useful during the initial scoping activities that you might undertake before planning or commencing your review. It may help you to identify key journal titles, appropriate keywords, and key authors. 

Cited reference searching is a technique that will help you locate additional articles for your research and allows you to identify more recent articles which cite the original article that you located. 

Here are the two multidisciplinary Manchester Met Library databases that allow you to carry out a cited reference search: 

• Web of Science has a cited reference search option. Go to Web of Science, change the basic search option to Cited Reference search and you can find articles that cite a person’s work. The suppliers of Web of Science have a video tutorial 
• Scopus allows you to search the references listed in an article

Google Scholar has a ‘Cited by’ feature, so you can see who cited a particular article or paper. 

The library run workshops on Cited reference searching.  See our Workshops page for further details and how to book on.  

Cited reference searching is a legitimate searching technique that will help you locate additional relevant articles for your research. Cited searching is a process of finding more recent articles that have cited relevant papers that have been identified within your main searches. 

Cited reference searches are supported in databases such as Scopus and Web of Science. If you would like further information about cited searching, the Library runs a cited reference searching workshop.

Research output comes in all shapes and sizes, and can vary in quality, some more robust and rigorous than others. 

An evidence hierarchy, often referred to in evidence-based medicine, provides a top-down scale whereby you first search for a recent well-conducted systematic review and if that is not available, then move down to the next level of evidence to answer your question. 

Evidence further down the scale may have a greater risk of bias. 

Definitions of grey literature vary, but it is generally accepted to mean any literature that is not formally published in conventional and established formats, such as books or journal articles. Examples of grey literature might include conference proceedings, working papers, reports from research groups, websites, pre-prints etc. 

It will depend on the purpose and scope of the review whether or not to include grey literature. Searching for grey literature can be a time consuming process, as you may need to search in many different places to find relevant material. Depending on your topic and available time, you may decide to search all or a limited range of grey literature. You may also need to record grey literature searching and results in a different way, and separately from other database searching. 

Inclusion and exclusion criteria are a list of pre-defined characteristics to which literature must adhere to be included in a study. They are vital for the decision-making progress on what to review when undertaking a systematic review and will also help with systematic literature reviews. 

You should be able to establish your inclusion/exclusion criteria during the process of defining your question. These criteria clearly demonstrate the scope of the study and provide justification for the exclusion of any information that does not meet these characteristics. 

Inclusion/exclusion criteria may cover the following aspects: 

Intervention, treatment, process or experience 
e.g. stage 4 lung disease patients or 

Setting 
e.g. community-based care 

Participants 
e.g. age, sex ethnicity etc. 

Reported outcomes 
i.e. whether the study’s reported outcomes are relevant to your study and have been presented objectively 

Age of study 
e.g. last 5 years 

Research methodology 
e.g. randomised control trial 

Sample size 
e.g. over 100 participants 

Place of study 
e.g. UK based 

Language 
e.g. English 

Type of publication 
e.g. primary research, peer-reviewed 

Hand-searching involves looking through the contents pages of journals, conference proceedings and abstracts. The purpose of hand-searching is to identify articles, which have not yet been included in databases or have not been catalogued or indexed. 

You can also find additional studies by looking through the references of papers that you have already identified in your review. Many databases will automatically display the references that a paper has used in their work, with relevant links to access the references. Some databases, like SCOPUS or Web of Science, will also contain an option for ‘Citied by,’ which will show you the papers that have cited them since they were published. 

There may be options within databases to save searches. Some databases, such as the EBSCO databases will allow you to create an individual account within the database and then you can save searches and your search history. For further information check the individual database help pages for how to save searches. 

You might also wish to review the section on documenting your search results. 

Most databases will allow results to be exported into EndNote and details of how each database does this can be found within the ‘Adding references to EndNote’ section of the EndNote guide. There will also be information in the help sections of the databases to explain how to export results. Some databases will allow results to be exported into an Excel spreadsheet, again check the individual database help pages for more information. 

One way to do this is to export the results into EndNote and then within EndNote it is possible to run a search to identify and remove duplicates. For further details look at the EndNote online handout (page 6) on the EndNote guide on the library website.