How to quality assure your dataHow to quality assure your data

Introduction

When you are an Editor of Research Output in Pure, you can implement a workflow to validate Research output metadata. The workflow allows you to assign responsibility for content input/validation to the right users, which makes sure the metadata and any uploaded files are as accurate as possible. 

What is QA in Pure?

The table below presents main points to consider to ensure the quality of your data in Pure.

All of the above (and often more), will typically be included in local institutional guidelines for the researchers.

Why is QA important?

The QA process helps ensure that the data in Pure is always valid and reliable.
This greatly improves:

Analysis of data

• Internal / external reporting
• Reliable institutional reports and analysis

Sharing of data, via

• Web service
• Reporting results to mail recipients
• Reporting to stakeholders
• Dashboard widgets

Exhibition of data, both

• Public – Pure Portal
• Internally in Pure

Workflow settings to consider

3 or 4 step workflow?

Pure gives you the option of defining the workflow with 3 or 4 steps. The difference between them is the number of QA steps (1 or 2). 
The steps are usually performed by three roles: Creator of Research Output, Editor of Research Output, and (in the 4-step workflow), Validator of Research Output. 
A separate role, Administrator of Research Output, can be enabled if you want to use it instead of the other roles. Unlike the Editor/Validator roles, the Administrator of Research Output is detached from the organizational hierarchy, which means there are no default tasks in workflows assigned to this role.

3-step workflow (Centralized)

This includes the following:

• Entry in progress
• For validation
• Validated

Which means there is one level of QA, handled by the Editor of Research Output.

The creator of the content, usually the researcher (or Submitter of research output), submits a Research output and can then either save it as "Entry in progress" (draft state) or complete the submission by moving it to the next step: "For validation". The Editor of Research Output can then make use of the workflow to mark it as "Validated". This workflow is often referred to as Centralized, as quite often the Library unit performs validation on behalf of the entire institution.

Below, you can see which users are responsible for which workflow step(s) in the 3-step process. Additionally, the colors indicate the scope of each role as it can be wider than the direct responsibilities.

Workflow step and responsible user (role)
Creator of Research Output	Creator of Research Output
	Editor of Research Output	Editor of Research Output

Note: It is also possible to move content back in the flow. For example, while Validator of Research Output has a task of moving content from "Approved" to "Validated", they can also send it back to the Editor of Research Output to re-do the approval ("For approval"), or even to the Creator of Research Output to modify the record and re-send the submission ("Entry in Progress"), essentially restarting the QA process.

4-step workflow (Decentralized)

This includes the following:

• Entry in progress
• For approval
• Approved
• Validated

This means there are two levels of QA:

• domain knowledge expert approval: e.g. a local user at a faculty (Decentralized) with expert knowledge of the domain approves the record
• validation: e.g. a librarian with expert knowledge on publishing policies and full text files etc validates it.

Note that in this case the Editor of Research Output does not perform the final step of QA. Instead an additional role, Validator of Research Output, is introduced which can perform the final step of QA. In other words, the Editor is now only responsible for the in-between step, "Approved".

Below, you can see which users are responsible for which workflow step(s) in the 4-step process. Additionally, the colors indicate the scope of each role as it can be wider than the direct responsibilities.

Workflow step and responsible user (role)
Creator of Research Output	Creator of Research Output
	Editor of Research Output	Editor of Research Output
		Validator of Research Output	Validator of Research Output

Note: It is also possible to move content back in the flow. For example, while Validator of Research Output has a task of moving content from "Approved" to "Validated", they can also send it back to the Editor of Research Output to re-do the approval ("For approval"), or even to the Creator of Research Output to modify the record and re-send the submission ("Entry in Progress"), essentially restarting the QA process.

Validation of external content

Apart from adding content produced at the current institution, researchers can provide information about outputs produced outside their current employment, which is particularly useful when working on Curricula Vitae (CVs). 
This is done by associating an output with an External Organization and making sure there are no links to any Internal Organizations in the Author and Affiliation part of the metadata. 
By default, such external records are not part of any standard workflow, which means you can save them in one step.

However, workflows can be enabled for external records so that they undergo the same QA procedure as internal content. 
You can enable external research output validation in Administrator > Research Output > Workflow step configuration > Change workflow step.

The illustration below shows an example of an external record with the single save button.

Revalidation

When content is in the final workflow step (Validated), it is by default locked for further modifications by submitters. This means that any requests for modification must be sent to either Editors or Validators. 
However, it is possible to unlock the content by enabling Revalidation. The change can be made in Administrator > Research Output > Workflow step configuration > Change workflow step.

When Revalidation is enabled, the submitters can make further modifications to the records, and the Editors/Validators responsible for those records will then either reject or accept the modifications made.

Note: It is best to unlock the validated content for a limited time period e.g. for final adjustments prior to a national assessment. Once the deadline is passed, revalidation can be toggled back to OFF.
 

Getting an overview

Editor tasks

A number of tasks are related to the process of QA of Research Outputs. Parts of the QA process can be identified via the 'Research Outputs' task. This task identifies the number of outputs that await Approval or Validation.

Filters to monitor and prioritize with

You can set up filter configurations to monitor different metadata to help prioritize tasks in the Research output listing. Below are some examples for inspiration:

Filter: Workflow

Use the Workflow filter to see which tasks are currently at a selected workflow step, and/or to differentiate between directions of workflow, i.e. see the tasks sent for validation by the creators of the content: "For validation - Creation/forwarding", or those submitted for re-validation: "For validation - Returning".

Note: These selections can also be reflected in your Task area.

Filter: Workflow duration

This allows you to focus on content remaining at a specific workflow step for a specified duration. For example, selecting a 'minimum of 7 days' filter will highlight content that has remained at a specific step for 7 days or more and may require action. This filter is best used in combination with the previous filter ('Workflow') to specify the step to monitor.

Filter: Period

This filter has different options. Used here is Content created date. This can be used to monitor how many Research outputs that created for example in the past week or month. Use in combination with other filters as 'Type' or 'Publication status' to narrow down to specific Research output types and/or in a specific publication state, e.g. Journal articles that are Published, may require faster validation. You can narrow further down to content published in specific Journals or submitted by specific Persons.

Filters: Workflow and Publication status

Using 'Workflow' and 'Publication status' filters in combination allows you to check for any irregularities. For example, it can highlight content that has been validated but has not been formally published yet.

Filters: Electronic version - file and SHERPA/RoMEO

Use this to monitor content published in journals/by publishers indexed with a Green SHERPA/RoMEO color and where no file yet has been uploaded. If you want to add full texts (or similar) for as many outputs as possible, this may give a hint of where a file can potentially be uploaded.

This can also be viewed in the Research output submenu 'Research Outputs with Electronic Versions' (See below), but using filters allows you to further narrow down your results.

Note: Even though the SHERPA/RoMEO colors are a good indicator of open access status, you should always check the relevant publishing policies as well.

List view vs. matrix view

Alter between list view and matrix view.

List view

• Allows to sort according to various criteria, such as 'most recently created'.
• Allows to view Favorites
• Allows for saved filter configuration

See LINK TO INTERFACE for more information.

Matrix view

• Lists publication states and workflow states on organizational levels the role is set to
• Filters on years
• Filters on internal vs. external affiliation

In the screenshot above all (internal) affiliations are listed with an aggregated count of the different publication states and the different workflow steps. Each number can be clicked to access a full list of content. For example, to view and work with all outputs for validation at the Department of Clinical Medicine, click 345.

Note: In Matrix view, the areas on the left and right of the screen are collapsed to make more room for the matrix. You can expand those areas by clicking small arrow icons on the edges of the screen.

Default menu points (Research Outputs)

Research Outputs menu contains a number of default sub-menu items:

Research Outputs with Electronic Versions

Here, you can view a matrix with information on electronic version availability for research outputs, including the Sherpa/RoMEO colour status. This can help identify where a full text file might be allowed to be uploaded. In the example, one green RoMEO color is listed as missing an electronic version.

The matrix can also help identify potential inconsistencies. In the example, there is a total of 6 outputs with electronic versions, and 2 without. However, only one output with an electronic version has a specified status (Embargoed) - which may indicate an oversight.