A computer program automates the process of selecting unbiased peer reviewers of research proposals submitted to NASA. Heretofore, such selection has been performed by manual searching of two large databases subject to a set of assignment rules. One database lists proposals and proposers; the other database lists potential reviewers. The manual search takes an average of several weeks per proposal. In contrast, the present software can perform the selection in seconds. The program begins by selecting one entry from each database, then applying the assignment rules to this pair of entries. If and only if all the assignment rules are satisfied, the chosen reviewer is assigned to the chosen proposal. The assignment rules enforced by the program are (1) a maximum allowable number of proposals assigned to a single reviewer; (2) a maximum allowable number of reviewers assigned to a single proposal; (3) if the proposing team includes a member affiliated with an industry, then the reviewer must not be affiliated with any industry; and (4) the reviewer must not be a member of the proposing team or affiliated with the same institution as that of a member of the proposing team.
This program was written Faiza Lansing and Anil Kantak of Caltech for NASA's Jet Propulsion Laboratory. For further information, access the Technical Support Package (TSP) free on-line at www.techbriefs.com/tsp under the Software category.
This software is available for commercial licensing. Please contact Karina Edmonds of the California Institute of Technology at (626) 395-2322. Refer to NPO-40902.
This Brief includes a Technical Support Package (TSP).
Automated Assignment of Proposals to Reviewers
(reference NPO-40902) is currently available for download from the TSP library.
Don't have an account?
Overview
The document titled "Automated Assignment of Proposals to Reviewers" from NASA's Jet Propulsion Laboratory outlines a systematic approach to assigning proposals to reviewers under specific rules to ensure fairness and objectivity in the evaluation process. The primary goal is to facilitate the proposal evaluation process, which is crucial for selecting projects that will receive funding for aerospace-related developments.
The document begins by introducing the problem of matching two sets of entities—proposals and reviewers—while adhering to certain assignment rules. It emphasizes that as the number of proposals and reviewers increases, manual assignment becomes impractical and error-prone. Therefore, an automated procedure is necessary to streamline the assignment process.
A matrix is utilized to represent the relationship between proposals and reviewers. Each proposal is assigned a row, while each reviewer occupies a column. The matrix is filled with "1s" and "0s," where a "1" indicates that a proposal can be assigned to a reviewer, and a "0" indicates that the assignment violates one or more established rules. The document details the rules governing these assignments, which include:
-
Reviewer Affiliation: If a proposal team includes members from an industry, reviewers affiliated with any industry cannot be assigned to that proposal. This rule aims to prevent potential bias in the evaluation process.
-
Conflict of Interest: Reviewers cannot be part of the proposal team or affiliated with the institution of any team member. This rule is designed to maintain the integrity of the review process.
-
Reviewer Limitations: There is a cap on the number of proposals a single reviewer can evaluate, ensuring that each proposal receives adequate attention and that reviewers are not overwhelmed.
The document also describes the implementation of a computer program using Microsoft Excel to automate the assignment process. The program systematically fills the matrix based on the defined rules, ensuring that the assignments are both efficient and compliant with the established guidelines.
In conclusion, this technical support package provides a comprehensive framework for automating the proposal review assignment process, highlighting the importance of fairness and objectivity in evaluating aerospace proposals. It serves as a valuable resource for understanding the methodologies employed by NASA to enhance the efficiency of proposal evaluations while adhering to ethical standards.
