Review Content TypeReview Content Type

1.1. Pre-requisites for the Review Type

As of v5.27.0 The Pure Review type is available to the following module users and under the following conditions;

• Subscribers to the Pure Award Management Module
• Subscribers to the UK REF Assessment Module (research output context)
• The review content type is under vendor control and made available on request via Pure Support. 

1.2. Description of the Review Type

The Pure Review Content Type allows enables institutions to collect, review and approve data in a structured fashion from a wide range of stakeholders. The feature has been initially developed as part of pre-award processes in the Award Management Module but has many wider uses across Pure and will be expanded over time into alternative areas. As from version 5.27.0 (June 2023) we have for example, additionally extended its use to support the review/assessment of research outputs for UK REF module subscribers. 

The use of the review type supports the development of reusable question templates by an administrator of review templates for a variety of needs (for example, risk management questionnaires, pre submission reviews and more) and allows the option of distributing question sets as content reviews to one or more reviewers as required, with all processing managed from one central location.  

Editors of Reviews can review individual or group responses as part of an approvals process and can use the data acquired via the review responses to further guide decision making and for the process of managing compliance. Reviewers can be selected from any internal Pure User – external (non-Pure user) reviewers are currently not supported.

The review content type introduces three new inter-related content types and several new roles required to manage the end-to-end review process. 

New Content types;

• Review Templates – Templates configured for reviewable content types, comprising multiple questions that can be applied to individual reviews.
• Reviews – Individual reviews that are requested by applicants/authors or created centrally by editors of reviews. The review editor assigns templates and adds the required respondents who are assigned to complete reviews
• Responses – Forms received by individual reviewers (respondents) with a request to participate in a review. 

New Roles

• Administrator of Review Templates (global) - role to create new templates as required
• Editor of Reviews (organisational) - role to create and configure reviews and act as gatekeeper for responses and final grading/approval.

Important note on roles: Due to content permissions, it is necessary for the Editor of Review role to be combined with the Editor of Content type roles in order to select items to review. If a user has the Editor of Reviews role alone they will not be able to search for an attach content items to the review - Hence, if an Editor of Reviews wants to set up a review for an application, they must also have the Editor of Application role permissions too. This allows access to the necessary information to associate with the review.   

1.3. Task  - Creating Review Templates (as Administrator of Review Templates)

1.3.1. Procedure - Creating review templates

Role required: Administrator of review templates 
 

Access to the review templates content type can be found in several places in Pure, dependent on the context of its use. For example, when enabled on both Award Management AND REF, the administrator of templates can browse to 

• Award Management->Review Templates
• REF2021->Review Templates

In both above cases, a filter is automatically applied to limit the templates to only those relevant for the content types in question. 

The review templates are also available via the editor tab, in which case no content filter is applied, and all types are shown.

Step 1. To create a new template, the administrator should browse to ‘review templates’ and click on the plus icon in the left-hand menu. A new template window will appear. 

Step 2. The administrator should add a Title for the template, the related content type it is intended to support and an optional description used to provide more information for the responder at the point of review. 

Step 3. The administrator can begin to add the questions to be used in the review. It is possible to add up to 100 questions per template if required. 

Questions can take several forms;

• Yes/No - a simple yes/no response with optional text description
• Text – text response (for long form answers)
• Choice – A multiple choice option. Administrators can add many different choices to select from, including an optional text description. 
• Rating – a configurable rating scale (5-10) for use in simple scoring, together with an optional text description.
• Classification – Responses requiring the reviewer to select from a defined classification scheme. Can comprise single or multiple selections, together with an optional text description

Note: For classification response questions, classification values are entered into the common scheme entitled "Review: Answers for classification based questions" with the URI syntax /dk/atira/pure/review/answers. Administrators must add the parent classification item to be used as the selectable classification scheme title and then the classification values to be use as the child values in that scheme. Remember that the child values must be related to the parent value within that scheme. 

For example;

Browse to master data → classification schemes.  Search for an edit the scheme  Review: Answers for classification based questions

Click - 'Add Classification' and enter the title for the classification scheme you wish to use

The, once saved, edit 'add classification' and add the classification values you wish to be included in this available selection, for example

Once, complete - choose 'add relation' on the title value for the classification scheme, and in the form that appears, choose the classification from the list (as shown in the screenshot below) to add as the child classification

Once complete for the added child classification, click 'create' and repeat for all classification values you wish to add as a relation to the parent. Once done, press save to complete the classification scheme update.  The resulting scheme should display the title and the related child classifications as per the following.

Step 4. Select whether a final grade scheme is required for the review, and if that grade scheme should be mandatory. 

Note: As per classification response questions above, classification values are entered into the common scheme entitled "Review: Final grades" with the URI syntax /dk/atira/pure/review/finalgrade. Administrators must add the parent classification item to be used as the selectable classification scheme title and then the classification values to be use as the child values in that scheme. Remember that the child values must be related to the parent value within that scheme. 

Once classification schemes are added, they are available for selection during the process of creating templates, or adding classification based questions to reviews when created manually. 

Result.

Once complete, and upon clicking 'save review question template' in the lower right hand corner of the screen, the template will be available for use on reviews as required.  The review templates will only be available for selection on the content types for which they are configured.  

1.4. Task - Creating Reviews (as Editor of Reviews)

1.4.1. Procedure - Creating new reviews

Role required: Editor of Reviews, Editor of [content type] 
 

Access to create the reviews can be found in several places in Pure, dependent on the context of its use. For example, when enabled on both Award Management AND REF, the Editor of Reviews can browse to 

• Award Management->Reviews
• REF2021->Reviews

Step 1. To create a new template, the Editor should browse to ‘reviews’ and click on the plus icon in the left-hand menu. A new review window will appear. 

Note: Navigating the Review Editor window

The review editor comprises four individual tabs, each of which support functionality required for the review process as per the following;

Description	Review Tab
Metadata Tab   The metadata tab provides the data relating to the review, the content it relates to and the reviewers assigned to complete the review tasks.    Managing organisation for each review is determined by the content added and will be indicated following the first save of the review.
Review Setup tab   The review setup tab is used to define the nature of the review. Review editors select the appropriate template to apply and may extend the template questions with additional questions and/or edit existing questions.    Documents may also be added based on the related content type.
Responses tab   The responses tab contains information on the summary and individual responses returned by the reviewer(s)    A summary tab provides a downloadable overview and individual response tabs provide a more detailed view
Finalize review tab   The finalize review tab contains the functionality to support the final grading and summary comment relating to the review and can only be completed when at least one response from the assigned reviewers has been received.    When required via the associated template, the tab also provides options to apply a final grade and indicate an overall approval status for the review.

Step 2. When in the metadata tab of the review, editors add the basic information requirements for the review, including the title for the review, description information, the related content being reviewed and who is expected to complete the review as a reviewer. Editors add a deadline date for the review and any reminders as required. 

Note: ‘Type’ dropdown in the review metadata tab. The type dropdown in the metadata tab allows institutions to apply categories to their reviews that are subsequently used to organise reviews and allow for selection and reporting in a number of different settings. The types are created via the review types classification scheme /dk/atira/pure/review/reviewtypes as per the following;

Step 3. When in the ‘review setup’ tab, the editor applies the chosen review template and, if required, updates the question details including the ability to add additional questions for individual reviews, plus edit the questions applied via the available templates. Note that only templates applicable for the content type to be reviewed are available for selection. 

Step 4. Once the details required in the metadata and review setup tabs are complete, the editor can either save the review to return to later, or click the ‘send to reviewers’ option in the lower right-hand corner of the review editor. Once this is selected, the review is sent to the individual reviewers to process. 

1.4.2. Note on form validation:

Certain fields on the review type are mandatory and have validation rules applied. If the Editor of Reviews attempts to save or send a review that does not meet the requirements, a validation tab will appear in the footer section of the review indicating the nature of validation issue, and will also indicate in red, the section that requires completion. 

1.5. Task - Creating Reviews (as personal user – applicant, or personal user - author)

1.5.1. Procedure - Creating reviews as personal user

Role required: Personal User 
 

Personal users acting as applicants on applications or as authors on research outputs (REF context) can propose a review on their content items. Editors of Reviews will receive notification of the review request and are then able to process the review or reject it if not required. 

Step 1. To create a new review request, the personal user should browse to their content item (application or research output) and to the following sections

Description	Applications
In the context of applications, the review option is available toward the foot of the application form in the relations section
The author selects from the available list of review types to categorise the review and a text note to provide further information for the review editor
Description	Research Outputs (REF Context)
In the context of research outputs, the review option is available in the left hand menu of the content type
The author selects from the available list of review types to categorise the review and a text note to provide further information for the review editor

Step 2. The Editor of Reviews will receive a notification mail and associated task informing them that a new review has been requested by a user as per the following;

The review is automatically created when requested by the user and is pre-configured with metadata taken from the content type. Fields pre-filled include the following

• Title
• Review request message (read only)
• Type
• Managing organisation
• Related content

Note: The Editor of Reviews has the option of rejecting the requested review in the event that the review request is deemed not required. Should the review be rejected, the review request will be clearly marked as such.

Step 3. The Editor of Reviews then proceeds to complete the remaining fields required for the review per the ‘Creating Reviews (as Editor of Reviews)’ task described above. 

1.6. Task - Responding to a review request

1.6.1. Procedure - responding to a review

Role required: User, Personal User 
 

Any user in Pure can be selected as a reviewer on a content type, except for system users (such as the root user). When added to a review, the Editor of Reviews selects the ‘Send to reviewers’ button and a review task and associated notification is sent to the named individuals.  Note that is possible for the creator of the content item (such as the applicant or author) to act as a reviewer. This is important use case that allows research administrators to use the review type in order to ask detailed questions about content items, such as for risk assessments, conflict of interest checks or other similar data gathering requirements. 

Step 1. The reviewer receives a notification indicating that they have a review to be completed. Notifications and email settings for reviews are available in the individual user settings and covers a variety of scenarios such as upcoming reviews, overdue reviews, new invitations and comments/questions raised about the response.

Step 2. Upon accessing the review, the review is asked to accept or decline the invitation. Only once the invitation is accepted will it be possible for the reviewer to complete the review questionnaire. 

Note: Due to issues of confidentiality, only a limited rendering of information about the item under review is provided, so it is necessary to include (where possible) associated documents, which can be linked to at the point of review setup. Where a review has an external link, such as DOI, it will be possible for the reviewer to click and access the content from its public location. 

Step 3. Once the review invitation is accepted, the user will complete the required sections as detailed and once all mandatory sections are completed the user is able to select the ‘submit’ option in the lower right-hand corner of the form. It is also possible for the user to save the review in a partial state to return to later. Once the review response is submitted, it is no longer possible to edit the review. If a new review is required, the reviewer will be required to contact the review editor to resend a new review.

1.7. Task - Reviewing responses and finalising a review

1.7.1. Procedure - finalising a review

Role required: Editor of Reviews

Note that reviews can only be finalised when at least one of the request reviewers have completed and submitted a response. If no responses have been received, then the editor will see the following message when browsing to the 'finalise review' tab

Step 1. Once the review request have been sent to reviewers and responses are received the review editor will be notified and can proceed to complete the 'finalise review' stage.  The finalise review tab is displayed as follows

Step 2. The review editor completes the summary and adds the final status option. In the event a final grade requirement has been added via the review template, then the review editor may select a value from the list.  Note that the review template may indicate that the final grade is mandatory in which case this must be completed before the review can be saved. 

Step 3. Once details are added, then the editor may select the 'finalise review' button in the lower right hand corner of the screen - once selected, a prompt to confirm is received. Clicking OK will complete the task and the review will now be marked as 'complete'. The status will be clearly displayed together with the approval finalisation status on the review overview screen and in the related content interface.

1.8. Task - Requests for information and commenting

1.8.1. Procedure - Adding comments and requesting further information from a reviewer

Role required: Editor of Reviews/User (reviewer)
 

Whilst the review is active, it is possible to send questions to the reviewers in the form of information requests. This feature allows the review editor to seek clarification on receipt of the review responses if necessary. The information request is available in the footer of the review editor and will allow questions to be added and directed to individual, or multiple reviewers as required.  Reviewers may reply using the similar information request option in the footer of their submitted review response.  

Step 1. To add an information request, the editor of reviews selects the 'add information request' option in the footer of the review form.  A pop up window appears to add the comment, and to select the reviewer or reviewers asked to respond

Once added, the information request is presented in the footer in the information request section as follows

Step 2. The notification is sent to the assigned user who is able to reply to the request directly. Note that the received of the information request is able to reply, but is NOT able to resolve the request. 

Step 3. Once the response has been received, the editor will be notified and may then proceed to either reply to ask further questions, or mark the request as resolved, indicating that the information provided is sufficient to proceed. The message indicator will be updated with a green 'resolved' marker as per the following.

1.9. Task - Handling exceptions

1.9.1. Procedure - Managing a declined review request

Role required: Editor of Reviews/User (reviewer)
 

In the event a reviewer chooses to decline a request for some reason, then they may do so. The review editor will be informed and can select to add a replacement reviewer as required. 

Step 1. The reviewer receives the invitation to review and selects the 'decline' option. They will be presented with a confirmation checkbox to indicate they wish to decline the review, enter a decline reason (Note: decline reason available from v5.27.1) and confirm. 

Step 2. The review editor receives a notification that a reviewer has declined the review. The status indicator for the reviewer updates on the review record to show the user as 'rejected' and the decline reasons (if entered) is displayed alongside

Step 3. The review editor may proceed to add a new reviewer as a replacement and the invitation will be sent to the newly added user once the review is saved. 

1.9.2. Procedure - Making the response summary text available to users

Role required: Editor of Reviews

The editor of reviews may choose to allow the requestor/author of the content type being review to see the summary text added when finalising the review.  By default, summary text will NOT be made available, nor will specific final grade information, although they will be able to see if the review is completed and whether or not the review has been approved (if options are selected). 

Step 1. On the finalise review screen, the review editor completes the summary text and indicates if they wish to allow the author/applicant to see the summary information

Step 2. Once the review is finalised and the summary text added, the summary text is available via the dropdown arrow option on the review referenced by the output.

1.9.3. Procedure - Rejecting and Deleting Reviews

Role required: Editor of Reviews

The editor of reviews may choose to delete a review or reject a review, depending on circumstances.   

Step 1. When choosing to delete a review, the editor selects from the delete option in the footer section of the review record.  Once the option is selected, the user is presented with a confirmation check and if selected, the record will be deleted entirely. Not that there is no option to restore a record once it is deleted.  The delete action will also remove all pre-existing responses made by reviewers and will delete uncompleted invitations to review from users. 

Step 2. The editor may also choose to reject a review request made by a user - if this option is selected, then the editor will be presented again with a confirmation screen and once confirmed the review will be retained but clearly marked as 'rejected'.  If there are outstanding invitations to review for the rejected record then they will be cancelled.