I recently encountered this when reviewing JBI's Manual for Evidence Synthesis, which notes:

"It is important to note that qualitative content analysis in scoping reviews is generally descriptive in nature and reviewers should not undertake thematic analysis/synthesis (i.e., JBI's meta-aggregative approach or meta-ethnographic approaches)."

The phrase threw me off balance.

JBI's guidance is considered the gold standard for scoping review methodology. Had I been doing it wrong all along? I remember using and citing thematic analysis in a couple of scoping reviews—and I'm not alone. A scoping review on the conduct and reporting of scoping reviews reveals that over 100 studies undertook formal qualitative analysis, including thematic analysis.

As I dug deeper, I realized that understanding how to analyze data in scoping reviews properly is crucial for their validity and utility. Here are three essential considerations every researcher should know.

Consideration # 1: Thematic Analysis Doesn't Make the Cut for Scoping Reviews' Purpose – Here's Why

According to Pollock and colleagues, advanced analysis methods such as thematic analysis aren't suitable for scoping reviews. The reason? These methods misalign with the purpose of scoping reviews.

Thematic analysis aims to provide insights regarding the meaningfulness of a phenomenon. When used in a review, it aggregates knowledge from a pool of studies to make sense of the phenomenon. In contrast, scoping reviews map existing evidence, identify key concepts, and determine research gaps—not to synthesize knowledge in the way systematic reviews do.

In short, the purpose of thematic analysis extends beyond that of scoping reviews, hence its inappropriateness.

Consideration # 2: The Simpler the Method, the Better

Basic descriptive data or content analysis aligns better with scoping reviews' purposes. According to Peters and colleagues, frequency analysis and percentages suffice for analyzing scoping review data.

Practical example: Tricco and colleagues exemplify this approach well in their article "A scoping review on the conduct and reporting of scoping reviews." Their analysis of 494 scoping reviews used simple descriptive statistics to characterize the methodological landscape of scoping review practices.

They reported frequency counts and percentages for key characteristics, such as the number of reviews that used a predefined charting form (43%) or conducted formal qualitative analysis (21%, including thematic analysis).

This straightforward descriptive approach effectively mapped the methodological landscape without attempting interpretative synthesis, perfectly aligning with scoping review methodology.

Consideration # 3: Still Need a Qualitative Approach? Adopt This 3-Phase Framework

Sometimes, basic content analysis becomes necessary when you seek to identify key characteristics related to a concept or wish to develop a conceptual framework based on literature.

For a practical qualitative analysis of scoping review data, Pollock and colleagues propose the following 3-phase approach:

Phase 1: Preparation
Decide whether to take a deductive approach (mapping data to an established framework) or an inductive approach (building a framework from the data).

Phase 2: Organizing
Structure data to answer your review question. This involves understanding evidence sources and linking their findings to your objectives.

The organizing process differs between approaches:

• Deductive: Extract data directly using your established framework

• Inductive: Develop a coding framework through open coding

Phase 3: Reporting
Combine visual presentations with supporting narrative to effectively communicate your findings.

Practical example: Chishtie et al.'s scoping review on visual analytic methods in health research used various visualization techniques, such as interactive dashboards, tree maps, and heat maps, to map research across geography, time, and sub-domains. The accompanying narrative contextualized these visuals without attempting to interpret patterns as evidence of effectiveness.

Data analysis in scoping reviews serves a distinct purpose: mapping evidence rather than synthesizing it. By selecting methodologically appropriate analysis techniques, we ensure that our scoping review maintains its integrity and provides readers with valuable orientation to the research landscape.

Remember, staying within methodological boundaries enhances credibility. For your next scoping review, consider these three key considerations to determine the appropriate analytic approach.

So, where should you look?

Understanding Data Extraction in Scoping Reviews

Contrary to systematic reviews, where you need to aggregate and synthesize the data from the papers' Results section, relevant information for your scoping review isn't confined to a single section of a research paper.

Each section of a scholarly article offers unique insights crucial to answering your research question.

Here is what the structure of a research paper looks like:

1. Introduction Section

   • Provides the conceptual framework and research rationale

   • Offers insights into how researchers define key concepts

   • Helps understand the theoretical background of the study

2. Methods Section

   • Explains the research approach and methodological choices

   • Offers details about how concepts were operationalized

   • Demonstrates the scientific rigor of the study

3. Results Section

   • Presents the primary findings

   • Provides empirical data and key outcomes

4. Discussion Section

   • Interprets the results in a broader context

   • Offers recommendations and practical implications

   • Highlights potential future research directions

It All Depends on Your Research Objective or Question

The first step in determining where to extract data is to clarify your research objective and review question. Scoping reviews aim to map the breadth of evidence available on a particular topic, concept, or issue. They also help identify key concepts, definitions, and research gaps.

Your objective will largely dictate which sections of a paper hold the most relevant data:

• If you are defining a concept, look at the Introduction or Method section, where authors often describe key terms and frameworks.

• If you are identifying recommendations, check the Discussion, where researchers provide interpretations, practical implications, and suggestions for future research.

• If you are seeking to understand what is known about a subject, the Results section is your go-to source.

Planning is Indispensable: Developing a Data Extraction Strategy

Following a structured approach is the key to ensuring consistency and completeness in your data extraction.

Here is the 3-phase approach based on Joanna Brigg Institute's guidance for scoping reviews:

1. Create a Charting Form

   • Design your extraction form using the PCC framework (Population, Concept, and Context). This ensures that you capture all essential elements related to your research question.

   • Consult JBI's guidance for scoping reviews to ensure your charting form aligns with best practices.

2. Pilot Test Your Extraction Form

   • Apply it to a few articles to check if it effectively captures the necessary information.

   • Collaborate with others when possible.

3. Revise, Revise, and Finalize

   • Revise the charting form for clarity and usability.

   • Document all changes and explain why such changes are necessary.

To ensure the comprehensiveness of the extracted data, use the check-in questions proposed by Pollock et al.

1. Was there anything missing from the extraction form?

2. Was there anything redundant included in the extraction form?

3. Was there anything on the extraction form that you did not understand or that could be further clarified?

4. Was there any unclear information in the accompanying guidance form?

—

Scoping reviews are an iterative process, and refining your approach to data extraction is a natural part of the journey. You can ensure a robust and efficient review process by aligning your strategy with your research objectives, using structured frameworks like PCC, and testing your charting form.

Enjoy the process, and don't hesitate to seek guidance if needed. Happy reviewing!

If it does, you're not alone.

Scoping reviews are essential tools for mapping concepts, theories, and evidence gaps from the literature. Yet even when you have a reliable compass—a solid protocol—in hand, the journey might still be filled with twists, turns, and hidden traps. An initially sound research question becomes unwieldy once evidence piles up. A seemingly reasonable search strategy yields only a handful of papers. To name a few.

These traps make the review process cumbersome, often requiring constant backtracking to set things right. To help you navigate more smoothly, this Research Log focuses on four common pitfalls lurking within scoping reviews and how to address them head-on.

Pitfall 1: Poorly Defined Research Question

The Problem

A good review starts with a clear research question.

This widely acknowledged recommendation may not be as straightforward as it sounds, especially when questions for scoping reviews don't need to be as specific as systematic reviews. But how broad should questions for scoping reviews be so that we don't wander aimlessly and lose track of the objective?

The problem with unclear research questions can start at the search for relevant papers and linger till the analysis phase. It makes your review feel disjointed and unfocused.

The Solution

To avoid this pitfall, revisit your review question with the PCC framework (Population, Concept, Context). By clearly defining the population, relevant concepts, and the context, the path toward your review objective will become clearer.

For example, instead of asking "What are the impacts of telehealth?" consider "What are the impacts of telehealth interventions (C) on rural older patients (P) receiving primary care services (C)?"

Key Takeaway: A well-defined research question is your compass—it keeps you focused and ensures your review stays manageable.

Pitfall 2: Inadequate Search Strategy

The Problem

Inadequate search strategies can lead to biased results, an inaccurate or incomplete review, and ultimately an unreliable foundation for future research. Poor search strategies may result from multiple factors: a vague research question, an incomprehensive set of search terms and keywords, limited database search, poorly constructed search strings, and syntax errors.

The Solution

To develop a comprehensive search strategy for our example of telehealth for older patients living in rural areas:

1. Adopt a trial-and-error approach

2. Develop search terms for each PCC element:

   • Population: "rural elderly" OR "rural older adult*" OR "rural senior*" OR "rural geriatric"

   • Concept: "telehealth" OR "telemedicine" OR "ehealth" OR "remote care" OR "virtual care"

   • Context: "primary care" OR "primary healthcare" OR "general practice" OR "family medicine"

3. Solicit help from librarians to refine these terms

4. Test and refine your search strings

The Joanna Briggs Institute's three-step approach can help develop a robust search strategy:

• Initial limited search in at least two databases

• Analysis of text words in titles/abstracts and index terms

• Second search using all identified keywords across all databases

Key Takeaway: Adequate search strategies balance relevancy and comprehensiveness, ensuring your review captures the full scope of available evidence.

Pitfall 3: Lack of Clarity in Inclusion/Exclusion Criteria

The Problem

Nothing diminishes the reliability of a scoping review like vague inclusion and exclusion criteria. Unclear and unspecific criteria lead to inconsistent application. One day, you include certain studies; the next day, you find those exploring similar concepts with the same population in a comparable context no longer fit for the review.

The consequence of these shifts is an arbitrary review that's hard to rely on for further research. When the scoping review is a team effort, unclear criteria result in screening disagreements, necessitating more time and resources to resolve.

The Solution

1. Start with our well-defined research question: "What are the impacts of telehealth interventions on rural older patients receiving primary care services?"

2. For each component, clearly specify the characteristics that will determine inclusion or exclusion

Example of clear criteria for our telehealth scoping review:

• Population: Adults aged 65+ living in rural areas

• Concept: Telehealth interventions delivered via video, telephone, or mobile applications

• Context: Primary care services provided by physicians, nurse practitioners, or physician assistants

• Types of evidence: Peer-reviewed research articles, government reports, and clinical practice guidelines

• Publication timeframe: 2010-2023

• Language: English

3. Pilot-test the criteria on a subset of retrieved studies

4. Refine based on pilot test results

Key Takeaway: Scoping reviews are an iterative process; changes are possible, but document and justify them with sound reasons.

Pitfall 4: Data Extraction Chaos

The Problem

Scoping reviews can be especially arduous when dealing with large volumes of data. Beyond the questions of what to extract, there's the big how-to question.

How do you extract data? What is the optimal process to ensure you have validatable data? Will you read and annotate the articles first, then organize data in a table later? Or will you fill data in the table as you go? Which tool can help to store, aggregate, and compare data?

These challenges can make the data extraction process chaotic and overwhelming, compromising the accuracy, consistency, and ultimately, the usefulness of the review.

The Solution

Here are strategies to navigate through this chaos for our telehealth example:

1. Develop a detailed data extraction tool (charting form) that includes:

   • Study characteristics: Author(s), year, country, study design

   • Population details: Age range, rural classification method, socioeconomic factors

   • Telehealth intervention: Technology used, frequency, duration

   • Primary care context: Provider type, care setting, integration with in-person services

   • Outcomes reported: Access measures, clinical outcomes, patient satisfaction, cost-effectiveness

   • Key findings related to the review question

2. Pilot-test the data extraction tool with other reviewers on 2-3 included studies to refine the form and ensure all relevant information is captured

3. Develop a software-assisted workflow for data extraction and management:

   • Choose the right tool (Excel, NVivo, Notion, etc.)

   • Create templates and standardized processes

   • Set up a system for regular backups and version control

Key Takeaway: A good workflow systemizes your process, preventing arbitrariness and increasing the reliability of the review. Software tools like Notion can be particularly useful for organizing and managing large volumes of data through automation capabilities.

Navigating Your Scoping Review Journey with Confidence

Scoping reviews serve as critical mapping tools in the research landscape, helping identify gaps, clarify concepts, and guide future investigations. While the pitfalls we've discussed—poorly defined research questions, inadequate search strategies, unclear inclusion/exclusion criteria, and data extraction chaos—may seem intimidating, they are entirely avoidable with proper planning and execution.

Remember that the cornerstone of a successful scoping review is clarity. Begin with a well-defined research question using the PCC framework. Develop a comprehensive search strategy with expert input when possible. Establish precise inclusion and exclusion criteria before you begin, and create a structured data extraction process that maintains consistency throughout your review.

Perhaps most importantly, embrace the iterative nature of scoping reviews. Unlike other methodologies that follow rigid protocols, scoping reviews allow for refinement along the way—provided you document your decisions and justifications transparently.

By avoiding these common pitfalls, you transform your scoping review from a daunting maze into a well-marked path toward valuable insights. The time invested in proper planning and methodical execution will pay dividends in the quality and usefulness of your final review.

What pitfalls have you encountered in your scoping review journey? Share your experiences in the comments below, and let's continue learning from each other's challenges and successes.

But what do you think would happen if the Americas didn't exist, and Columbus's fleet sailed on? Would they have reached Asia? Or would they have run out of supplies and felt extremely discouraged after months with no sight of land?

To avoid such a mishap, anyone who wishes to reach their research destination must conduct literature reviews. Think of literature reviews as essential preparation before you head out on an expedition. You wouldn't sail to unknown lands without a map or some understanding of what that journey entails.

Literature reviews help you understand what is already known about a topic, what you should do to make a contribution to the field, where you should invest time and energy, and which areas you should avoid to prevent redundancy.

While why we conduct literature reviews is straightforward, what we need to do when conducting one is less clear. Despite the voluminous literature on the subject, I rarely see courses or formal training on literature reviews. (If you know of any good ones, please share—I would love to enroll!)

In this Research Log issue, I want to lay out a 6-phase approach that will help you start your journey with a clear mind on where you want to go, what you want to achieve, and what steps you should take to get there.

Let's start, shall we?

Phase 1: Know Thyself and Thy Research Purpose

To guarantee the success of your literature review and avoid feeling lost and wasting effort (like Columbus might have if America wasn't there!), it's important to know both your destination and the resources you have available.

Ask yourself these fundamental questions:

1. What is the purpose of this literature review? Your research goal will help narrow down the type of literature review you conduct. If you wish to write clinical guidelines for telehealth practitioners, you will opt for systematic reviews rather than narrative reviews.

2. What resources do I have for this literature review? Resources can be your time, research expertise, or a team that provides necessary support along the way. You wouldn't want to conduct a meta-analysis if statistical skills aren't your strength and you work mostly alone.

Vapiro and colleagues also recommend considering your epistemological orientation when planning a literature review. According to them, you should choose a method that suits your view of what knowledge is. If you seek objective, quantifiable evidence, conducting an experiential qualitative review is not a good fit.

Phase 2: Understand the Typology of Literature Reviews

I think we all feel overwhelmed by all the existing literature reviews. The last time I checked, there were 25 methods of synthesizing knowledge!

In this Research Log issue, I'll stick with the 8 types proposed in Vapiro and colleagues' model. These types are frequently used in health science research:

Note: For further guidance on each type of review, consult the Literature Review Series of the Journal of Graduate Medical Education.

Phase 3: Match Your Research Purpose with the Type of Literature Review

It's time to align your research purpose (Phase 1) with the appropriate type of literature review (Phase 2).

Think of a literature review as a lens through which you examine and capture the essence of a body of research. Just as a photographer wouldn't use a telescope for a macro-image of an ant nest, you wouldn't want to use a narrative review to synthesize evidence for clinical guidelines on a specific health intervention.

Consider these additional factors when making your choice:

• Which type of literature review is more common in your research field?

• If the type that best fits your purpose isn't commonly used in your field, how can you justify your choice?

Phase 4: Use a Research Protocol

Whether you opt for a systematic or non-systematic review, a detailed protocol will help maximize rigor and minimize bias. It provides a roadmap for the review team and serves as a base for the audit trail. Even when you are the lone reviewer, it helps you stay focused on the research goal.

While systematic reviews come with well-established protocols, you'll need to use more judgment when developing one for flexible types of reviews, such as narrative reviews. Start with these four basic elements:

1. State the research purpose

2. Define the inclusion criteria

3. Plan the search strategy

4. Outline the analysis methods

Phase 5: Build a Support Circle to Maximize Your Literature Review

Even on a solo expedition, you can always solicit help to ensure a safe journey. The same applies to conducting literature reviews.

Ask yourself whose input and opinion will enhance the quality of your review. For example:

• Librarians at your university are experts in search strategies—consult them to refine yours

• Subject matter experts are valuable resources

• I've contacted researchers via ResearchGate to ask for access to their papers and clarification on specific points

Phase 6: Experiment and Document

A good plan is an adjustable plan. Your initial research question might change after a preliminary review of the literature. Such deviations from the original protocol necessitate detailed documentation and justification.

Be prepared to adapt to emerging evidence and document any departures from your protocol.

Remember: Like Columbus, you're setting sail into partially charted waters. With the right preparation, clear goals, and appropriate methods, your literature review will help you discover new insights—even if they're not exactly what you set out to find!

Your go-to tool is Excel. You have all the necessary data: articles' metadata and the excerpts neatly organized by themes. Everything looks so clean. The code structure is ready to apply to other articles.

Not long after, when you read and code the next article in line, things get off track. The excerpts from this article don’t quite fit in with the intended code. Two options lay out in front of you:

1. Change the wording of the code so that it can better represent the idea drawn from the excerpts

2. Create a new code for these excerpts and revise the excerpts in the existing codes to check if other excerpts fit better with the new one.

Does this scenario sound familiar to you?

If you’re like me and my colleagues, we often have to weigh between options and change the code content in analyzing qualitative data. Performing qualitative analysis is far from a straightforward process. It necessitates multiple revisions involving re-categorizing excerpts and restructuring codes.

This iterative process is fundamental to qualitative analysis, but it requires a system that can:

• Flexibly accommodate changes in code wording and excerpt categorization

• Automate updates across your dataset to prevent errors

• Maintain a clear record of all modifications

Here is where a Notion system comes in and surpasses the Excel spreadsheet. Often the go-to tool in literature reviews, Excel cannot offer you the above features. If you want to use Excel anyway, well, you might end up where we were in the past: spending dozens of hours going back and forth between Excel and annotated files to validate information. And despite all the hard work, we couldn’t escape the errors.

The previous newsletter issue has prepared you with the necessary mindset and knowledge to understand how using Notion databases to analyze literature review data can save you time and free you from headaches and frustration in analyzing content in a literature review.

If you have created a Notion account, keep reading. If not, go to the Notion website to create one. It takes you no more than a minute to finish the registration.

The process of building a Notion system for an efficient literature review involves the following three steps:

• Step 1: Create a database triad for articles, extracts, and codes

• Step 2: Add properties for each database to create a comprehensive view of data entries

• Step 3: Connect and aggregate data

Step 1: You only need three databases for your literature review

While we can create as many databases as we wish, three databases are the right number for data management and manipulation. With less than three, you have to stuff your database with too much information, making it harder to manage, manipulate, and transform data. With more than three, the setup might be redundant and unnecessarily complicated.

To capture all essential elements of the reviewed articles and ease your analysis work, let’s create these three databases:

1. The Article database to hold information on all articles selected for your review

2. The Extract database to store relevant excerpts from the selected articles, including originally associated themes

3. The Code database to store and carry out your analysis work.

While the Article and Extract databases hold native information from original articles, i.e., information you extract, instead of summarizing or paraphrasing, the Code database is where you use your own words to capture the meaning of the extracts.

Follow this process to create a database in Notion.

1. Open a new Notion page. Let’s name it the Literature Review Project. Of course, you can name it however you like as long as it makes sense to you and your colleagues.

2. In the first line, type /database to call out the menu for database formats. You can choose any option. For simplicity, let’s go with the first choice: Database inline.

3. In the New View window, choose: New table| Start a new blank database.

4. Give your new database a descriptive name. I opt for Articles DB.

Use the same steps to create databases for the extracts and the codes.

Step 2: Add database properties to create a comprehensive picture of the data

A property in Notion databases has one single job: adding context to the database’s entries. Similar to columns in Excel spreadsheets, properties enable you to organize information in categories, making it easy to filter and sort data.

For advanced data entry and screening, it is important to choose the right property type. Database properties in Notion can be of various types, each responding to a specific data need. For example, to track your working progress with an article, you can use the Status property. To tag articles with multiple topics of interest, use the Multi-select property instead of the Select property. With the Multi-select, you can assign multiple tags to a single article while the Select property restricts you to only one tag, making it useful for single-category categorization.

Here is the list of properties for the three databases in a regular literature review.

To add a property to a database, you can either click on the + sign in the table view of the database or select Properties on the top right of the database, then click on + New property.

For example, here are the two steps to add a property like Year of Publication:

1. At the top right of the database, select Properties, then New Property.

2. In the new drop-down menu, choose Number

3. Give the property a descriptive name like Year of Publication

Do the same with the rest of the properties in the lists.

Step 3: Time to establish database relationships

To save time and avoid errors and frustrations during data analysis, you must connect the Article, Extract, and Code databases. Linking these databases enables data automation, preserving their connections, thereby reducing errors during data manipulation. To accomplish the task, all you need is the Relation property.

Set up the connection between the Extract database and the Article database

It’s time to use the Connection Map that we created together in the previous issue. This map serves as a blueprint to set up the relation type between databases. If you need to do a quick check, go back to read more about the Connection Map here.

You can create the relation either from the Extract database or the Article database. For the sake of illustration, let’s go to the Extract database and add a new property.

1. In the drop-down menu Property type, opt for Relation.

2. In the Related to menu, choose the database that you want to link with: the Article database.

3. In the Edit property menu:

   1. Set the Limit to 1 page. By default, the relation type between two databases is set as No limit. That means one entry in the Extract database can link to multiple entries in the Article database. However, we all know that one extract belongs to one and only one article. Setting up the right relation type between these two databases will help avoid such an error.

   2. Turn on the Two-way relation button so that the connection is automatically added to the Article database.

Note: You don’t need to change the Relation type in the Article database since it is set as No limit by default, preserving the characteristics that one article can have multiple extracts.

Set up the connection between the Extract and Code databases

In the Extract database, create a new Relation property.

1. In the Related to menu, choose the Code database.

2. In the Edit property menu

a. Set the Limit to 1 page

b. Turn on the Two-way relation button so that this connection is automatically added to the Code database.

What about the Rollup property? It’s also in the Connection Map

Like Relation, Rollup is a connection property in Notion. It displays and summarizes information from a related database. For example, to see the original categories assigned to articles’ excerpts, a Rollup property in the Article database can pull data from the Original Category column in the Extract database.

Use the Connection Map to see what property to pull from a linked database and the following process to roll up data. For illustration, I’ll pull the Code property from the Extract database in the Article database:

1. In the Article database, add a new column/property and choose Rollup from the Property type menu.

2. In the Relation line, choose the related database with the data you want to roll up. It’s the Extract database in this case.

3. In the Property line, choose the Code property.

4. In the Calculate line, choose Show unique values. In this way, no matter how many times a code is attributed to an article’s extracts, it will show up once. For more information on types of Rollups, check out this page.

Follow the same process for the rest of the properties marked as Rollup in the Connection Map.

You’re all set. Now it’s time to fill in data, intersect, and restructure these raw pieces to extract gems from them!

If you have any questions about this setup, feel free to drop me a comment or send me a message. I’ll get back to you as soon as possible.

If you read my latest post about Notion as a free, easy-to-use, yet powerful qualitative analysis tool, and you are tempted to try it, this post is for you.

It will prepare you with the essential mindset and knowledge to build a simple yet powerful Notion database setup for a rigorous, trustworthy analysis of your literature review data.

Mindset preparation: Why Notion?

Everyone knows data analysis is conducted to answer a research question. But what does it mean concretely? What do you want to obtain? And how does using a qualitative analysis tool simplify this process?

Whether you know it intuitively, or through my earlier post on the benefits of using Notion in analyzing the content for a literature review, you want a tool or a setup that helps:

• Link extracts to articles, codes, and sub-codes or categories, and automate these connections so that you don't have to go back and forth to change them manually.

• Provide dynamic views so that you can look at data from different angles, understand them better, revise code structure, develop themes, etc.

A good Notion database setup offers all these features at no cost. It ensures:

1. The inherent connections between Articles - Extracts

2. The acquired connections between Articles/Extracts and Codes and Categories

It also offers various views of data, so that you can examine the following:

1. What are the salient themes regarding your specific question?

2. What do the code/categories look like when articles are grouped by certain criteria?

Knowledge preparation

Understand basic Notion features

Think of Notion databases as a super advanced version of Excel spreadsheets.

Like Excel spreadsheets, Notion databases are data containers. Each database contains a collection of entries called pages. Pages within a Notion database are defined by properties that act like columns.

For example, the Article database of a literature review holds selected articles as pages. Each entry represents a single article. Common properties that help define an article include:

• Title

• 1st Author

• Year of Publication

• Journal, Research Question or Purpose

• Methodology

• etc.

Here is how a data table for your systematic literature review looks in Excel and its equivalent in Notion.

Notion databases surpass spreadsheets in two ways:

1. They enable connections between databases. With the Relation and Roll-up properties, you can link pages in the Article database to those in the Extract database and view any property in the connected database with one click.

2. They offer diverse dynamic views of data. While Excel spreadsheets only present data in tabular form, you can change the way you want to view the data, either as a table or a Kanban board with each theme as a card.

2 data questions to answer before sailing out for a smooth content analysis

Answering these questions will help you understand what to include in your databases and how you should set up relations between them. Ultimately, it will facilitate your analysis and save you time in the long run by removing any redundant back-and-forth.

What essential elements do you want to capture from the selected articles?

List out all the elements. They help identify which databases to create and what properties to include in each. Here is an example of the common elements that you want to capture for a systematic literature review:

1. Info about the article: Author, Year of Publication, Journal, etc.

2. Info about the study: Methodology, Measures/Instruments, Sample Size, Sample Age, etc.

3. Articles’ extracts relevant to the Research Question

4. Codes & sub-codes or categories.

How are these elements related to each other?

Do some elements belong to the same group/category? What is the nature of the relation between articles, extracts, and codes? One article can have many extracts, but one extract cannot belong to more than one article. What about the relation between, say, extracts and codes? Can we assign one extract to multiple codes? Or is it essential that one extract belongs to one and only one code?

Map out all potential connections. This practice will make it easier to set up properties and relations between databases.

The connection map between elements in your literature review might look like this.

Action!

Create your Notion account

If you already have an account in Notion, skip this step. If not, no need to fret. The process is straightforward. It’ll take you 2 minutes at most.

Plus, all the Notion features you need are offered in a free plan. What can be better than this?

Here is the link to create your Notion account: https://www.notion.com

Welcome onboard!

With the Notion account, you’re now ready to set up a system for efficient literature review analysis.

Notion – the tool that helps manage literally everything in your life: track new habits, oversee the workflow, capture notes, document the progress, etc. This tool can also save you from feeling overwhelmed, lost, and frustrated, typical emotions you experience in qualitative analysis.

Indeed, I wonder if the Notion people considered data analysis, particularly qualitative analysis, when they positioned their platform as "all-in-one workspace". Given the pricing of high-end qualitative analysis software such as NVivo or Atlas.ti and the bulkiness of Excel as well as its lack of automation, when you choose to stay with a low-cost solution, I wholeheartedly believe that Notion will be a game changer for your academic life.

It can do pretty much what NVivo or Atlas.ti offer you.

Its Plus plan which offers more teamwork features is free for academia.

Plus, it is truly "all-in-one workspace," where you can also track the work progress and manage other logistic details. More on this topic will need to wait for a later post. For today's, let's focus on how Notion can help you conduct qualitative analysis with rigor and save you from all the hassles.

What exactly do we need from a qualitative analysis tool?

To better understand how Notion can help you, let’s take a moment and think about what we really need from a qualitative analysis tool.

Let's say you conduct a systematic literature review, a project that often involves a great deal of content analysis. The first tool that comes to your mind might be Excel. Everybody uses it. There's no learning curve nor bad surprises when you collaborate with others.

Except one thing: No matter how you want to treat extracted text data, you have to do it manually.

Excel is made to work with numbers, not text. If you put a lot of text there, you'll lead yourself to paralysis mode, unable to function afterward.

To use Excel for the content analysis of a systematic literature review, one possible simple workflow would be:

1. Annotate articles in Zotero

2. Tag extracts with codes

3. Put codes in Excel.

With one article having multiple codes, you have to create multiple columns to hold codes to make things readable. This solution is workable when your data pool is small, and you have a well-defined code structure. The story changes dramatically once you have dozens of articles, and need to revise your code structure several times.

Excel has automation functions as mentioned in one of my previous articles on using Excel's Table format for literature review. But the automation in Excel only works with number data.

In a systematic literature review, we need to work with loads of text. And text data requires different operations.

Specifically, to analyze the content of the articles selected for the review means to bring out their connections, how the articles converge in one aspect and diverge from the other in another. For rigor and trustworthiness of the analysis, we want to be able to trace back what is said in one article and validate if it does refer to whatever code we assign to it.

It means…

We want to be able to an extract to the article that contains it.

We also want to be able to assign a code to this extract, and make sure that this code shows up in the article in question.

If we change our mind about a code, say, modify its wording to better capture the meaning of content, we want this change to be synchronized across the whole dataset.

In short, we want the functions of a qualitative analysis software like NVivo. Better if it is user-friendly and doesn't cost an arm and a leg.

Well, if that’s what you’re looking for, you are in the right place. Welcome to the wonderland of Notion!

Here is what Notion can offer you:

1. Link extracts to articles, codes and categories & automate data connections

With a good Notion database setup, you can easily link extracts to articles, modify the code assigned to extracts, and see changes synchronized everywhere.

Notion’s database function and its two properties "Relations" and "Rollups" are essential for this setup. One of Notion’s most powerful and flexible features, databases enable you to create structured collections of information. For your literature review project, you might want to create the following databases:

1. Article database to hold information of all articles selected for your review

2. Extract database to store relevant extracts from the selected articles, including associated themes as indicated in the original articles

3. Code database to store codes, sub-codes, and carry out your analysis work.

To establish connections between these databases, you will need to set up "Relations" and "Rollups" properties. While the "Relations" property allows you to link items like extracts from the Extract database to articles in the Article database and to codes in the Code database, "Rollups" helps aggregate, for example, Codes assigned to Extracts in the Article database.

That means, when you go to a specific article, you not only see all its extracted data, but you'll also see all the codes linked to these extracts. Equally, when you modify the wording of a code, this change will show up automatically in the Extract view and Article view.

Want to read more about these properties and see how they work? You can have a look at Notion’s website.

2. Obtain dynamic views of data in a couple of clicks

While Notion databases come with multiple views, enabling you to look at your data from various angles, I find the kanban board view is the most important display that we should care about when conducting qualitative analysis.

Organizing data items into columns with each entry as a card, the board display helps visualize your code structure in a dynamic way.

Let's say, you have a two-level code tree made up of codes and sub-codes.

With a board view that groups sub-codes by codes, you can see what sub-codes make up a specific code. Furthermore, since all the databases are linked together, with one click you can show what extracts from which articles were assigned to those codes and sub-codes.

One image is worth a thousand words. Here is how a board view of codes and sub-codes looks like.

3. Work and share findings with others at ease

My clients might be in a better position to tell you what they think when I share the work progress and the preliminary findings of our literature review project using Notion databases. But I can at least tell you what I felt.

It's not the first time I've needed to share my analysis with someone to get their input. But it's the first time I've felt relieved.

I didn't bombard my clients with bulky data that would require them hours to navigate through.

Myself neither. I didn't need to spend hours tearing my hairs out for a convivial way to share data with them.

With a couple of clicks, I can change the view of the database setup that I've used for data analysis. And voila, I have a clean board view to present to my clients.

That’s it, my friends. That’s Notion’s wonderland. I’m still amazed with all what I can do with this single tool.

And let’s not forget: Notion is free.

Use it.

Optimize it.

Make your life easier with it.

I bet it’s data overload.

After screening thousands of articles, you end up with dozens of eligible ones, each containing a wide range of data to extract beyond the study's findings: The first author, their domain, the year of publication, the place of study, the methodology, to name a few.

To store, organize and do basic descriptive analysis, Excel is the go-to tool for its ease of use. However, working with a large data set in Excel can be challenging if you don't know how to optimize the tool.

Imagine discovering several new relevant articles; this means hours of incorporating new inputs in your data set and the frustration of redoing the calculations and analysis manually.

But what if Excel automatically update the data set and the analysis for you?

The table format, one of Excel’s most underused features, can help you do exactly that. By the Table format, I don't mean wrapping your dataset with borders manually. What I mean is to convert automatically a data range in a neatly formatted table with a clear header row.

Here are 3 reasons why you should consider use the Excel table format the next time you conduct a literature review.

Excel tables ensure consistent formatting

Once you press Ctrl + T to format a data set as a table, the data will be seamlessly organized into well-defined rows and columns.

The header row stands out with bold characters and remains visible as you scroll down. Imagine you need to review dozens of articles. You no longer need to scroll up and down to keep track of column names when inspect the middle and last part of the data set.

As consistent formatting is applied across the range of data, you can inspect data efficiently. For example, when all data in a specific column follow the same pattern, any deviation will stand out immediately.

Excel tables help accelerate data cleaning process

Cleaning data is time consuming. There is no question about that.

However, the Table format can streamline the process with built-in sort and filter features.

While sorting helps you quickly identify outliers and duplicates, filtering helps spot errors and check the consistency of column data. For instance, applying the filter to the "Study Location" column can quickly reveal variations like U.S., U.S.A., and U.S. with extra spaces.

Using the Table format, you can also apply validation rules to restrict the type of data entered or set value limits for a data range. For example, if the minimum population age in your literature review is 65, setting an age validation rule will trigger an error if a number below 65 is entered.

Excel tables optimize the analysis

Here are three ways the Excel table format can save you time and free you from frustration when analyzing literature review data.

1. Dynamic range

Let’s stay with the scenario where you must include several articles to the corpus of studies.

Without the Table format, it means that you have to incorporate data and do all the calculations again. Manually.

Here comes one of the greatest benefits of converting your data into a table: It transforms your data set into a dynamic range.

Concretely, it means:

• Your table will automatically expand each time you enter new data.

• All formulas, charts, and PivotTables will be updated without any manual adjustments.

Can you see all the headaches it saves you from?

2. Structured references

Structured references means that you can use table and column names to refer to data instead of cell addresses. That makes it easier to call out data from specific columns and add them in your formulas.

For example, instead of typing =AVERAGE(E2:E50) to calculate the average age of the studied populations, you can simply point the cursor to the Population age column to complete the function =AVERAGE. The formula =AVERAGE(Table1[Population age]) is also easier to read and understand.

3. Integration with analysis and visualization tools

Whether you want to conduct a descriptive analysis of the selected studies or create a dynamic visual to demonstrate the trends in the corpus of knowledge, the Table format offers you features to accomplish the tasks in an instant.

First, you can obtain quick summaries of the data set with PivotTables, one of Excel's powerful features that provides you insights in totals, averages, counts and other aggregate metrics.

Second, with the Table format, you can use Power Query to transform the data to your specific needs. This feature enables you to:

• merge and split columns

• create calculated fields

• group data based on specific columns

• calculate both basic and advanced statistics like sum, average, median, mode, standard deviation, percentiles.

Third, with the Table format, hence access to PivotCharts, you can create dynamic and interactive charts. Your charts are not only always up to date thanks to the dynamic range feature. You can also explore and gain insights from the changes in the charts by adjusting the filters. Imagine your PivotChart shows the number of studies published each year. By using the Slicers feature, you can

• Filter by Country: Select one or multiple countries to see how the number of studies in these countries has changed over time.

• Filter by Methods: Curious to see how number of studies adopting certain methods changed over time? Well, the chart can show you exactly what you are looking after a click.

So how can we convert a bundle of data into table?

There are at least 3 ways you can use to create a table from your dataset.

Way # 1: Use the Insert tab

• Select the data range

• Go to Insert tab

• Click on the Table button in the Tables group.

• A Create Table window will appear, asking you to confirm the data range. Check the "My table has headers" if the data has a header row.

• Click Ok.

Way # 2: Use the Table format in the Home tab

• Select the data range as in the Way # 1

• In the Home tab, choose the Table format.

• Choose a table style

• The Create Table dialog box appears. Do as in the Way # 1.

Way # 3 (my favorite): Use shortcut keys

• Place the cursor anywhere in the desired data range

• Press Ctrl + T

• When the Create Table dialog box appears, do as in the Way # 1.

It's that simple. Now, all your data are organized into a formatted table.

The benefits mentioned here extend beyond literature review data. You can always apply the Table format to any data set in Excel. It will you save you time and spare you from headaches.

Without clean data, your analysis might be erroneous, biased, and lead to no actionable insights.

That is why data cleaning might take up a majority of your workload as data analyst.

While there are multiple tools and techniques to ensure data’s tidiness, junior data analysts often struggle to do a thorough cleaning. You might have some idea of what needs to be cleaned and why it is important. But you still wonder: Where should I start? How can I be sure that data are ready to be analyzed?

For a more complete and systematic cleaning, here is a checklist that could help you rest assured that things are under control.

But first, what is data cleaning?

To clean data is to transform them from raw state into consistent state, so that they fit for analysis. In other words, with this process, you want to ensure that data are: 1) free of technical errors, and 2) consistent with the domain knowledge.

For a breakdown, it means two things:

1.    Technically speaking, data must be stored in a comprehendible format with recognizable column names and in the right format. For example, values in date columns must be in the Date type, not the String type.

2.    Regarding domain knowledge, data must conform to constraints based on knowledge of the subject. An example of violation might be a row indicating that a child of five years old holds a driver's license, which contradicts the law of driving age. In this phase, you also deal with missing values and outliers.

Checklist for thorough data cleaning

To ensure the technical correctness of data and their consistency with the domain knowledge, you must thoroughly inspect data, develop a plan for cleansing operations and, when needed, justify why you choose to handle an error in a certain way.

This checklist aims to help beginners proceed the cleansing process with more confidence, knowing where to start and when to stop.

1. From raw to technically correct data

Data are technically correct when they are stored in a comprehendible format like table or data frame with recognizable column names and in the right type.

So, here is a list of questions to inspect data’s technical correctness.

1.1 Are data in a readable format like table for data frame?

If the answer is No, you must convert data into understandable forms.

1.2 Are data stored with recognizable column names?

In other words, can you tell what variable the field represents just by looking at the column name? If not, change the column name to a clearer one. Without a clear understanding of data, you will have trouble in analyzing them.

1.3 Are the observations unique?

Put differently, do data contain duplicates? Duplicates lead to faulty analysis. Fortunately, it is easy to identify and remove duplicates in either Excel or R.

1.4 Are data in the right type?

Data can be a number, a string of characters, a Boolean, or a date. Errors occur when values of a certain type are stored in another. For example, miscalculation happens when values in a numeric field are stored as text.

1.5 Are data in a consistent format?

This question refers to the uniformity of data. Are all data in a column stored in the same format?

There are multiple ways of writing dates, numeric numbers, phone numbers, etc. No matter what format is chosen, make sure that it is used consistently.

Here are several checkpoints to ensure that data are in consistent formats:

• Date format: Is the date format used uniformly across the data set? While dates in many parts of the world are often written as DD-MM-YYYY, date data from the United States might be written as MM-DD-YYYY. So, be careful.

• Case check: Are all values in text columns written in the same case? When one category is written both as annual member and Annual member, expect the error in your calculations. So, choose one format and apply it through the whole data set.

• Text format: Do all values in text columns following a specific standard like phone numbers or postal codes have the same formatting? Make sure that all postal codes are stored consistently as XXX-XXX or XXX XXX, instead of both ways.

1.6 Are data free of white spaces?

A neglected white space before or after values can lead to misinterpretation, miscalculations, and other complications when merging data. A tidy data set has no place for white spaces.

2. From technically correct data to analyzable data

Once data are technically correct, you must guarantee their readiness for the analysis by ensuring that:

1. data conform with domain knowledge, and

2. missing values, obvious errors and outliers are handled.

You can ask three following questions to inspect the analyzability of the data.

2.1 Does data correspond to real-world knowledge about the subject?

Data must be consistent with constrains based on knowledge of the domain. It will be handy to verify predefined rules for the data set. For example, a rule that the Age column must fall in a certain range helps identify those out of this range.

Here are some checkpoints to inspect data’s consistency with real-world knowledge.

• Data range check (for integral data): Do values fall within expected range?

• Category check (for categorical data): Do categorical columns have correct and consistent categories? Are different entries used for the same category?

• Length check (for text or string data): Do text columns with specific standards have consistent string lengths? The postal code for Canada has six characters. Therefore, any value with under or over six is erroneous.

2.2 Are there any missing values?

Data can be missing randomly or systematically. You can choose to remove them all at once or fill in the blank. No matter what operation you choose, make sure you can justify it with the domain knowledge.

2.3 Are there any outliers in the data set?

Though outliers are not necessarily errors, they can skew your analysis. Hence, a justification based on domain knowledge is needed whether you choose to remove or keep them.

Sources

1. de Jonge & van der Loo (2013). An Introduction to data cleaning with R. Available at: https://cran.r-project.org/doc/contrib/de_Jonge+van_der_Loo-Introduction_to_data_cleaning_with_R.pdf

2. Data cleaning checklist by DataCamp: https://www.datacamp.com/blog/infographic-data-cleaning-checklist

3. Vadali (Dec, 2017). Day 7: Data cleaning — All you need to know about it. Available at: https://becominghuman.ai/day-7-data-cleaning-all-that-you-need-to-know-about-it-23b05738abe7